Jul 08 03:19:43.831208 ip-10-0-129-190 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 03:19:44.236019 ip-10-0-129-190 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 03:19:44.236019 ip-10-0-129-190 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 03:19:44.236019 ip-10-0-129-190 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 03:19:44.236019 ip-10-0-129-190 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 03:19:44.236019 ip-10-0-129-190 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 03:19:44.237878 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.237777    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 03:19:44.243087 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243067    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:44.243087 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243083    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:44.243087 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243087    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:44.243087 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243090    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:44.243087 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243094    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243097    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243100    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243103    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243106    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243109    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243112    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243115    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243118    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243120    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243123    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243125    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243127    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243130    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243132    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243135    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243137    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243140    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243142    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243145    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:44.243265 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243152    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243155    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243157    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243159    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243162    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243164    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243167    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243169    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243172    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243174    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243177    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243179    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243184    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243188    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243193    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243196    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243199    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243202    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243205    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:44.243759 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243207    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243210    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243213    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243215    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243218    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243221    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243223    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243226    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243229    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243231    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243233    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243236    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243238    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243241    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243243    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243246    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243248    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243251    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243253    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:44.244234 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243257    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243260    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243262    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243266    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243269    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243272    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243275    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243278    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243282    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243286    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243290    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243292    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243295    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243298    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243300    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243302    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243305    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243307    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243310    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243312    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:44.244708 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243314    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243318    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243321    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243323    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243325    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243328    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243330    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243332    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243335    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243337    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243340    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243342    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243345    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243347    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243349    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243352    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.243354    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244056    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244062    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244065    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:44.245200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244068    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244071    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244074    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244077    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244079    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244082    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244085    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244087    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244090    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244092    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244095    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244097    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244100    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244102    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244105    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244108    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244110    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244113    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244115    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244117    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:44.245688 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244120    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244123    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244125    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244129    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244133    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244136    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244138    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244140    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244144    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244147    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244150    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244154    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244160    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244163    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244165    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244169    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244171    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244174    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244176    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:44.246205 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244179    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244182    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244184    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244194    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244197    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244200    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244204    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244208    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244212    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244214    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244217    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244219    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244222    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244224    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244227    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244229    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244231    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244234    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244236    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244239    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:44.246696 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244241    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244244    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244247    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244249    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244252    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244257    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244259    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244262    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244264    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244266    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244269    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244272    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244274    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244277    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244279    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244283    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244286    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244288    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244291    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244293    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:44.247195 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244295    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244299    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244301    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244304    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244306    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244309    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244311    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244314    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244316    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244319    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244321    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244323    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244326    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244328    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244331    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244333    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244336    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244691    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244695    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244698    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:44.247666 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244700    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244703    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244705    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244709    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244711    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244714    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244717    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244719    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244722    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244725    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244727    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244729    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244732    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244735    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244738    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244741    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244743    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244745    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244748    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244750    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:44.248157 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244753    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244755    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244758    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244760    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244763    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244765    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244767    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244770    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244773    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244777    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244779    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244782    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244784    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244786    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244789    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244812    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244815    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244818    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244822    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:44.248634 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244825    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244828    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244831    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244833    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244836    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244839    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244841    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244844    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244847    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244849    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244851    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244854    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244856    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244859    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244861    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244863    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244866    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244868    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244870    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244873    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:44.249104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244877    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244880    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244883    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244887    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244890    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244894    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244896    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244899    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244901    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244903    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244906    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244908    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244912    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244915    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244918    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244921    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244923    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244926    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244928    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244931    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:44.249577 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244934    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244936    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244939    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244941    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244944    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244960    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244963    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244966    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244968    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244971    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244973    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244976    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244980    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244982    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244985    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244987    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.244990    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245053    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245059    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245068    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245074    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 03:19:44.250074 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245083    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245088    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245093    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245099    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245103    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245106    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245109    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245112    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245115    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245118    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245121    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245124    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245128    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245131    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245135    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245137    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245140    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245143    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245146    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245150    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245153    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245156    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245159    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245163    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 03:19:44.250560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245166    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245169    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245172    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245174    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245178    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245181    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245183    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245186    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245189    2580 flags.go:64] FLAG: --enable-server="true"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245191    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245196    2580 flags.go:64] FLAG: --event-burst="100"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245198    2580 flags.go:64] FLAG: --event-qps="50"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245201    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245204    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245207    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245211    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245214    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245217    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245219    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245222    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245225    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245228    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245230    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245233    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245236    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 03:19:44.251141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245238    2580 flags.go:64] FLAG: --feature-gates=""
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245242    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245244    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245247    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245250    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245253    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245256    2580 flags.go:64] FLAG: --help="false"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245259    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245263    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245266    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245268    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245271    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245275    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245277    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245280    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245282    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245285    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245288    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245291    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245294    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245297    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245300    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245302    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245305    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 03:19:44.251778 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245308    2580 flags.go:64] FLAG: --lock-file=""
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245310    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245313    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245315    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245320    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245323    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245326    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245329    2580 flags.go:64] FLAG: --logging-format="text"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245331    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245334    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245337    2580 flags.go:64] FLAG: --manifest-url=""
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245339    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245343    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245346    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245350    2580 flags.go:64] FLAG: --max-pods="110"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245352    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245357    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245359    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245362    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245365    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245368    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245370    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245377    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245380    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245383    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 03:19:44.252482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245386    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245389    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245393    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245396    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245399    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245401    2580 flags.go:64] FLAG: --port="10250"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245404    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245407    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08f24c033dc9bbe60"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245410    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245413    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245415    2580 flags.go:64] FLAG: --register-node="true"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245418    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245422    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245425    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245427    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245430    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245434    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245437    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245439    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245442    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245445    2580 flags.go:64] FLAG: --runonce="false"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245448    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245451    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245453    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245457    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 03:19:44.253099 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245460    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245463    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245466    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245468    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245471    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245474    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245476    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245479    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245482    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245484    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245487    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245492    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245494    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245497    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245500    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245503    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245505    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245508    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245511    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245513    2580 flags.go:64] FLAG: --v="2"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245517    2580 flags.go:64] FLAG: --version="false"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245521    2580 flags.go:64] FLAG: --vmodule=""
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245524    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.245527    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245635    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:44.253685 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245638    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245641    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245644    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245647    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245650    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245652    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245656    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245659    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245662    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245664    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245667    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245669    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245672    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245674    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245677    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245680    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245682    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245685    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245687    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245689    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:44.254299 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245692    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245695    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245698    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245701    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245703    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245706    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245708    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245711    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245714    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245719    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245722    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245725    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245728    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245731    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245737    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245740    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245742    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245745    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245749    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:44.254776 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245751    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245754    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245757    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245759    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245762    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245764    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245767    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245769    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245771    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245774    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245777    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245779    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245782    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245784    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245786    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245789    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245792    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245796    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245798    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245801    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:44.255283 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245804    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245806    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245808    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245811    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245814    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245816    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245819    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245823    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245825    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245828    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245830    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245834    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245836    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245839    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245841    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245843    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245846    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245848    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245850    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245853    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:44.255768 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245855    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245857    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245860    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245862    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245864    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245867    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245869    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245872    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245874    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245877    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245879    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245882    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245884    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245886    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245890    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245894    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245899    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245903    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:44.256335 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.245907    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.246591    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.253331    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.253349    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.254181    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.254188    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254250    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254254    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254257    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254260    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254263    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254266    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254269    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254272    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254275    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:44.256757 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254278    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254281    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254283    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254286    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254288    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254291    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254293    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254296    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254299    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254301    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254304    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254307    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254309    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254312    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254314    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254317    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254319    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254321    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254324    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254327    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:44.257142 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254336    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254339    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254342    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254344    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254346    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254349    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254351    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254353    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254356    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254358    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254361    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254363    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254366    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254369    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254371    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254374    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254376    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254378    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254381    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254383    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:44.257614 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254386    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254388    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254391    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254393    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254396    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254398    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254400    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254403    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254405    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254408    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254410    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254414    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254418    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254427    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254430    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254434    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254438    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254441    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254444    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:44.258104 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254446    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254450    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254453    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254456    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254458    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254461    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254463    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254466    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254468    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254471    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254474    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254476    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254479    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254481    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254483    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254486    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254488    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254490    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254493    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254495    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:44.258559 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254498    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254500    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254502    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254505    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254508    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254511    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254513    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254523    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254525    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254528    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254530    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.254535    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254667    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254670    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254673    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:44.259060 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254676    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254678    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254681    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254684    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254687    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254691    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254693    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254696    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254698    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254701    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254703    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254706    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254708    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254711    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254713    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254716    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254718    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254720    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254723    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:44.259426 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254725    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254727    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254730    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254733    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254735    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254738    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254746    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254748    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254751    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254753    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254756    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254758    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254761    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254763    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254765    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254767    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254770    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254773    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254776    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:44.259886 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254779    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254782    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254785    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254787    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254790    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254792    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254794    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254797    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254799    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254801    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254804    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254806    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254808    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254811    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254813    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254816    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254819    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254821    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254824    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254827    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:44.260406 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254836    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254838    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254840    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254843    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254845    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254847    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254850    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254852    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254855    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254858    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254860    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254863    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254865    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254867    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254870    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254872    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254874    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254877    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254879    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254881    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:44.260888 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254884    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254886    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254888    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254891    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254893    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254895    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254898    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254900    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254903    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254905    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254907    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254910    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254912    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254920    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254922    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254925    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254927    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:44.261376 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:44.254929    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.254934    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.255067    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.257033    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.258488    2580 server.go:1019] "Starting client certificate rotation"
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.258601    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.258655    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 03:19:44.261775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.261045    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 03:19:44.262502 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.262481    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 03:19:44.273386 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.273368    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 08 03:19:44.275258 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.275243    2580 log.go:25] "Validated CRI v1 image API"
Jul 08 03:19:44.276796 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.276781    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 03:19:44.279238 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.279220    2580 fs.go:135] Filesystem UUIDs: map[0400efb5-dcb1-4e41-8ce6-7c446383b91b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 80e547c9-f90f-45b4-8992-0b47d2e5149a:/dev/nvme0n1p3]
Jul 08 03:19:44.279294 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.279239    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 03:19:44.285082 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.284977    2580 manager.go:216] Machine: {Timestamp:2026-07-08 03:19:44.28370483 +0000 UTC m=+0.325889490 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100927 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec296ec00e327a7f914ce682ce18ae47 SystemUUID:ec296ec0-0e32-7a7f-914c-e682ce18ae47 BootID:02cb2fc1-e586-4af1-96fa-3915c8ca2f3a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:1a:ec:a0:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:1a:ec:a0:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:af:8f:13:11:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 03:19:44.285082 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.285078    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 03:19:44.285189 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.285151    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 03:19:44.288370 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.288343    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 03:19:44.288497 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.288374    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-190.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 03:19:44.289278 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.289268    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 03:19:44.289315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.289280    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 03:19:44.289315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.289287    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 03:19:44.289368 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.289353    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 03:19:44.289853 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.289842    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 03:19:44.291168 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.291158    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 03:19:44.294030 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294019    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 03:19:44.294142 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294133    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 03:19:44.294180 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294165    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 03:19:44.294180 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294179    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 03:19:44.294235 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294200    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 03:19:44.294235 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294214    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 08 03:19:44.294235 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294226    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 03:19:44.294651 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294639    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 03:19:44.294683 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.294666    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 03:19:44.296773 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.296758    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 03:19:44.298480 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.298468    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 03:19:44.298521 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.298498    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 03:19:44.299526 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299514    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 03:19:44.299564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299530    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 03:19:44.299564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299537    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 03:19:44.299564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299544    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 03:19:44.299564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299554    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 03:19:44.299564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299563    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 03:19:44.299715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299573    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 03:19:44.299715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299580    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 03:19:44.299715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299588    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 03:19:44.299715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299610    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 03:19:44.299715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299630    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 03:19:44.299715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.299635    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 03:19:44.300349 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.300337    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 03:19:44.300349 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.300349    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 03:19:44.303509 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.303495    2580 server.go:1268] "Started kubelet"
Jul 08 03:19:44.303609 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.303569    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 03:19:44.307634 ip-10-0-129-190 systemd[1]: Started Kubernetes Kubelet.
Jul 08 03:19:44.308306 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.308083    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 03:19:44.308306 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.308154    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 03:19:44.308306 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.308284    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-190.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 03:19:44.308751 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.308610    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-190.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 03:19:44.309063 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.308862    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 03:19:44.309863 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.309834    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 03:19:44.310759 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.310732    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 03:19:44.311164 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.311148    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 03:19:44.312707 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.311769    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-190.ec2.internal.18c03252f424a83b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-190.ec2.internal,UID:ip-10-0-129-190.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-190.ec2.internal,},FirstTimestamp:2026-07-08 03:19:44.303466555 +0000 UTC m=+0.345651217,LastTimestamp:2026-07-08 03:19:44.303466555 +0000 UTC m=+0.345651217,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-190.ec2.internal,}"
Jul 08 03:19:44.317180 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317166    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 03:19:44.317248 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317181    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 03:19:44.317303 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.317249    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.317810 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317791    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 03:19:44.317810 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317790    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 03:19:44.317930 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317820    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 03:19:44.318008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317966    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 03:19:44.318008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317976    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 03:19:44.318008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317979    2580 factory.go:55] Registering systemd factory
Jul 08 03:19:44.318008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.317993    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 08 03:19:44.318357 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.318325    2580 factory.go:153] Registering CRI-O factory
Jul 08 03:19:44.318357 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.318344    2580 factory.go:223] Registration of the crio container factory successfully
Jul 08 03:19:44.318480 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.318393    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 03:19:44.318480 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.318420    2580 factory.go:103] Registering Raw factory
Jul 08 03:19:44.318480 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.318450    2580 manager.go:1196] Started watching for new ooms in manager
Jul 08 03:19:44.319520 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.319403    2580 manager.go:349] Starting recovery of all containers
Jul 08 03:19:44.329645 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.329618    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 03:19:44.329766 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.329743    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-190.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 03:19:44.329881 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.329868    2580 manager.go:354] Recovery completed
Jul 08 03:19:44.334192 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.334178    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.336480 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.336456    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.336543 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.336484    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.336543 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.336494    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.337007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.336993    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 03:19:44.337093 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.337007    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 03:19:44.337093 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.337026    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 03:19:44.337093 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.337028    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.337812 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.337798    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.337894 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.337827    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.337894 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.337844    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.338300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.338288    2580 policy_none.go:49] "None policy: Start"
Jul 08 03:19:44.338344 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.338304    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 03:19:44.338344 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.338315    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 03:19:44.339095 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.339081    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p5279"
Jul 08 03:19:44.339462 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.339452    2580 policy_none.go:47] "Start"
Jul 08 03:19:44.346234 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.346172    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-190.ec2.internal.18c03252f61c4705  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-190.ec2.internal,UID:ip-10-0-129-190.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-190.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-190.ec2.internal,},FirstTimestamp:2026-07-08 03:19:44.336471813 +0000 UTC m=+0.378656476,LastTimestamp:2026-07-08 03:19:44.336471813 +0000 UTC m=+0.378656476,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-190.ec2.internal,}"
Jul 08 03:19:44.346791 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.346775    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p5279"
Jul 08 03:19:44.377789 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.377607    2580 manager.go:337] "Starting Device Plugin manager"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.377817    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.377830    2580 server.go:87] "Starting device plugin registration server"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.378078    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.378090    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.378197    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.378281    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.378291    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.378809    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 03:19:44.388660 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.378858    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.450202 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.450169    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 03:19:44.451577 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.451549    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 03:19:44.451677 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.451597    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 03:19:44.451677 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.451626    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 03:19:44.451677 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.451665    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 03:19:44.454409 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.454390    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:44.478607 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.478588    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.479398 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.479383    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.479467 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.479416    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.479467 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.479427    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.479467 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.479449    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.486487 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.486446    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.486487 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.486466    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-190.ec2.internal\": node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.513152 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.513124    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.552372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.552343    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"]
Jul 08 03:19:44.552490 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.552412    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.553334 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.553320    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.553405 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.553350    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.553405 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.553360    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.554641 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.554629    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.554751 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.554736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.554785 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.554775    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.555329 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.555314    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.555401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.555338    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.555401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.555348    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.555401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.555364    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.555401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.555381    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.555401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.555389    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.556566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.556547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.556685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.556572    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:44.557229 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.557213    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:44.557311 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.557246    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:44.557311 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.557261    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:44.585490 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.585472    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-190.ec2.internal\" not found" node="ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.589865 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.589849    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-190.ec2.internal\" not found" node="ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.613485 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.613471    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.619826 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.619809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd44f3e11daee88facac900a13ae9553-config\") pod \"kube-apiserver-proxy-ip-10-0-129-190.ec2.internal\" (UID: \"bd44f3e11daee88facac900a13ae9553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.619899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.619840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa31c6dd93bd9d145e38cc39ff6f3247-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal\" (UID: \"fa31c6dd93bd9d145e38cc39ff6f3247\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.619899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.619865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa31c6dd93bd9d145e38cc39ff6f3247-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal\" (UID: \"fa31c6dd93bd9d145e38cc39ff6f3247\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.714008 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.713977    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.720345 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.720330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa31c6dd93bd9d145e38cc39ff6f3247-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal\" (UID: \"fa31c6dd93bd9d145e38cc39ff6f3247\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.720424 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.720355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa31c6dd93bd9d145e38cc39ff6f3247-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal\" (UID: \"fa31c6dd93bd9d145e38cc39ff6f3247\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.720424 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.720372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd44f3e11daee88facac900a13ae9553-config\") pod \"kube-apiserver-proxy-ip-10-0-129-190.ec2.internal\" (UID: \"bd44f3e11daee88facac900a13ae9553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.720424 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.720417    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd44f3e11daee88facac900a13ae9553-config\") pod \"kube-apiserver-proxy-ip-10-0-129-190.ec2.internal\" (UID: \"bd44f3e11daee88facac900a13ae9553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.720543 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.720449    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa31c6dd93bd9d145e38cc39ff6f3247-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal\" (UID: \"fa31c6dd93bd9d145e38cc39ff6f3247\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.720543 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.720448    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa31c6dd93bd9d145e38cc39ff6f3247-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal\" (UID: \"fa31c6dd93bd9d145e38cc39ff6f3247\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.814776 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.814718    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:44.889219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.889183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.893163 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:44.893015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:44.915574 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:44.915553    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.016181 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.016148    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.116656 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.116595    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.202084 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.202053    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:45.217177 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.217157    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.257555 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.257535    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 03:19:45.257918 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.257640    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 03:19:45.257918 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.257679    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 03:19:45.318040 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.318018    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.348822 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.348794    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 03:14:44 +0000 UTC" deadline="2028-01-06 20:54:19.894264086 +0000 UTC"
Jul 08 03:19:45.348822 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.348821    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13145h34m34.545447363s"
Jul 08 03:19:45.359114 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.359098    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:45.370460 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.370409    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:45.372137 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.372121    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:45.372234 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.372153    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:45.372234 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.372166    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:45.419087 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.419062    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.455149 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:45.455110    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd44f3e11daee88facac900a13ae9553.slice/crio-78606b74b7b009cd1fa8c34ec717a878458647eae8101959f2f0aabb2b88c84b WatchSource:0}: Error finding container 78606b74b7b009cd1fa8c34ec717a878458647eae8101959f2f0aabb2b88c84b: Status 404 returned error can't find the container with id 78606b74b7b009cd1fa8c34ec717a878458647eae8101959f2f0aabb2b88c84b
Jul 08 03:19:45.455555 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:45.455532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa31c6dd93bd9d145e38cc39ff6f3247.slice/crio-74825a974b8ba1f0c11029b696e11de09665750d7e75a6f03555649bfb77d3be WatchSource:0}: Error finding container 74825a974b8ba1f0c11029b696e11de09665750d7e75a6f03555649bfb77d3be: Status 404 returned error can't find the container with id 74825a974b8ba1f0c11029b696e11de09665750d7e75a6f03555649bfb77d3be
Jul 08 03:19:45.461323 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.461298    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:19:45.519160 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.519132    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.619536 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:45.619515    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-190.ec2.internal\" not found"
Jul 08 03:19:45.699568 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.699498    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:45.718268 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.718247    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:45.731762 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.731744    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 03:19:45.732961 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.732940    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal"
Jul 08 03:19:45.738930 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:45.738919    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 03:19:46.294163 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.294134    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:46.295175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.295156    2580 apiserver.go:52] "Watching apiserver"
Jul 08 03:19:46.302894 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.302871    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 03:19:46.304306 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.304275    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-dflvf","kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5v465","openshift-multus/multus-additional-cni-plugins-b5ppb","openshift-multus/multus-q5l55","openshift-network-operator/iptables-alerter-bjx6v","openshift-ovn-kubernetes/ovnkube-node-hrq66","kube-system/global-pull-secret-syncer-txl57","kube-system/konnectivity-agent-6zmpz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq","openshift-dns/node-resolver-sz6hp","openshift-image-registry/node-ca-qwssj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal","openshift-multus/network-metrics-daemon-8cxvd"]
Jul 08 03:19:46.306523 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.306498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.308356 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.308330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.308832 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.308812    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 03:19:46.309081 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.309060    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.309327 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.309293    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 03:19:46.309417 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.309328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.310095 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.310073    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 03:19:46.310190 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.310121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.310395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.310372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nls8h\""
Jul 08 03:19:46.310815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.310760    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.310815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.310792    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.311071 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.311054    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 03:19:46.311401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.311385    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9nbsk\""
Jul 08 03:19:46.311709 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.311689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.312158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.312135    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.312447 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.312427    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.312545 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.312433    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5j9m7\""
Jul 08 03:19:46.313372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.313352    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.313833 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.313815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kjxsp\""
Jul 08 03:19:46.313919 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.313870    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 03:19:46.313919 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.313914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 03:19:46.314564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.314544    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.314776 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.314757    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 03:19:46.314958 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.314942    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.315147 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.315127    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.315529 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.315508    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 03:19:46.315856 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.315708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kjj22\""
Jul 08 03:19:46.316203 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.316180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.317718 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.317695    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.321248 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.321228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rnds2\""
Jul 08 03:19:46.321484 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.321466    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 03:19:46.321962 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.321946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.322272 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.322254    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 03:19:46.322533 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.322518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 03:19:46.323479 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.323459    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 03:19:46.324136 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.324114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.324230 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.324211    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:46.325271 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.325246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.325368 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.325332    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:46.326033 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.326001    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8kctr\""
Jul 08 03:19:46.329005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.328196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.329880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-systemd-units\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.329905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.329930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-var-lib-kubelet\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.329946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-host\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.329962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.329986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-etc-kubernetes\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvpsr\" (UniqueName: \"kubernetes.io/projected/be3ebbcc-0f5c-4d38-bde8-008358c5b341-kube-api-access-gvpsr\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2r85\" (UniqueName: \"kubernetes.io/projected/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-kube-api-access-t2r85\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b78k\" (UniqueName: \"kubernetes.io/projected/b289c98c-c888-4672-9b09-03e12ae66c52-kube-api-access-4b78k\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-cni-netd\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-env-overrides\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2cd6863-1b67-4db0-b559-6d957315a9f9-tmp\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-os-release\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-run\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-slash\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-ovn\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-dbus\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-system-cni-dir\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53f61516-48cb-43e6-95e1-f0b62350a522-agent-certs\") pod \"konnectivity-agent-6zmpz\" (UID: \"53f61516-48cb-43e6-95e1-f0b62350a522\") " pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-ovnkube-script-lib\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5796db27-474f-4e9e-9b0a-a3a219c600b4-tmp-dir\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccdhc\" (UniqueName: \"kubernetes.io/projected/f2cd6863-1b67-4db0-b559-6d957315a9f9-kube-api-access-ccdhc\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-host-slash\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.330824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-log-socket\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-kubernetes\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-cni-bin\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-iptables-alerter-script\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-cnibin\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be3ebbcc-0f5c-4d38-bde8-008358c5b341-cni-binary-copy\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndf8m\" (UniqueName: \"kubernetes.io/projected/5796db27-474f-4e9e-9b0a-a3a219c600b4-kube-api-access-ndf8m\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysconfig\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysctl-d\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-kubelet\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330908    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-run-netns\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.330947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-systemd\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.330969    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vsjx\" (UniqueName: \"kubernetes.io/projected/ed399293-2342-4edf-bf8b-2b333a686a3e-kube-api-access-9vsjx\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5796db27-474f-4e9e-9b0a-a3a219c600b4-hosts-file\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.331694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-modprobe-d\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-os-release\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-ovnkube-config\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-system-cni-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-node-log\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fd4m\" (UniqueName: \"kubernetes.io/projected/b2f91405-83ae-4cc0-a508-d93ff9d99c56-kube-api-access-2fd4m\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-kubelet\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed399293-2342-4edf-bf8b-2b333a686a3e-ovn-node-metrics-cert\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-sys\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-cni-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-daemon-config\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cnibin\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-socket-dir-parent\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-k8s-cni-cncf-io\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.332632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-netns\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-hostroot\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-conf-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53f61516-48cb-43e6-95e1-f0b62350a522-konnectivity-ca\") pod \"konnectivity-agent-6zmpz\" (UID: \"53f61516-48cb-43e6-95e1-f0b62350a522\") " pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysctl-conf\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331930    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.331948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-systemd\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-lib-modules\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-tuned\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-kubelet-config\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-cni-multus\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332153    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332170    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-var-lib-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-etc-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-cni-bin\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cni-binary-copy\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-multus-certs\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.333621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.332452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g2rp7\""
Jul 08 03:19:46.335407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.335364    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 03:19:46.336321 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.336303    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zhvw9\""
Jul 08 03:19:46.337371 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.337352    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 03:19:46.337614 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.337597    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 03:19:46.340286 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.340267    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 03:19:46.365823 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.365803    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-556fx"
Jul 08 03:19:46.386936 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.386917    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-556fx"
Jul 08 03:19:46.419006 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.418983    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 03:19:46.432505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2r85\" (UniqueName: \"kubernetes.io/projected/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-kube-api-access-t2r85\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.432610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b78k\" (UniqueName: \"kubernetes.io/projected/b289c98c-c888-4672-9b09-03e12ae66c52-kube-api-access-4b78k\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.432610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.432610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4kbw\" (UniqueName: \"kubernetes.io/projected/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-kube-api-access-p4kbw\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.432610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-cni-netd\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-env-overrides\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2cd6863-1b67-4db0-b559-6d957315a9f9-tmp\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-os-release\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-run\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-socket-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-registration-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-slash\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-ovn\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-dbus\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-system-cni-dir\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53f61516-48cb-43e6-95e1-f0b62350a522-agent-certs\") pod \"konnectivity-agent-6zmpz\" (UID: \"53f61516-48cb-43e6-95e1-f0b62350a522\") " pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.432817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-ovnkube-script-lib\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5796db27-474f-4e9e-9b0a-a3a219c600b4-tmp-dir\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccdhc\" (UniqueName: \"kubernetes.io/projected/f2cd6863-1b67-4db0-b559-6d957315a9f9-kube-api-access-ccdhc\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-host-slash\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.432950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433008    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-run\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-log-socket\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-kubernetes\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-cni-bin\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-iptables-alerter-script\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-cnibin\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be3ebbcc-0f5c-4d38-bde8-008358c5b341-cni-binary-copy\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndf8m\" (UniqueName: \"kubernetes.io/projected/5796db27-474f-4e9e-9b0a-a3a219c600b4-kube-api-access-ndf8m\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysconfig\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysctl-d\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-kubelet\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.433517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-run-netns\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433335    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-cni-netd\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-systemd\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vsjx\" (UniqueName: \"kubernetes.io/projected/ed399293-2342-4edf-bf8b-2b333a686a3e-kube-api-access-9vsjx\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5796db27-474f-4e9e-9b0a-a3a219c600b4-hosts-file\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-modprobe-d\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-os-release\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-serviceca\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-ovnkube-config\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-system-cni-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-node-log\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fd4m\" (UniqueName: \"kubernetes.io/projected/b2f91405-83ae-4cc0-a508-d93ff9d99c56-kube-api-access-2fd4m\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-kubelet\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed399293-2342-4edf-bf8b-2b333a686a3e-ovn-node-metrics-cert\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-sys\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.434376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-cni-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-daemon-config\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433893    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-env-overrides\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-device-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.433997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cnibin\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-socket-dir-parent\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-k8s-cni-cncf-io\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-netns\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-hostroot\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-conf-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53f61516-48cb-43e6-95e1-f0b62350a522-konnectivity-ca\") pod \"konnectivity-agent-6zmpz\" (UID: \"53f61516-48cb-43e6-95e1-f0b62350a522\") " pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysctl-conf\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434277    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-systemd\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-lib-modules\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-tuned\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-kubelet-config\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.435264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-cni-multus\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-etc-selinux\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-var-lib-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-etc-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-cni-bin\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cni-binary-copy\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-multus-certs\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-sys-fs\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434726    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-os-release\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434737    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-log-socket\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434750    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-os-release\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn6n6\" (UniqueName: \"kubernetes.io/projected/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-kube-api-access-dn6n6\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434822    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-kubernetes\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434831    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-cnibin\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.434859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-host\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435407    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435435    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-ovnkube-config\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435470    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-cni-bin\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435496    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-iptables-alerter-script\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435514    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be3ebbcc-0f5c-4d38-bde8-008358c5b341-cni-binary-copy\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435525    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-system-cni-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.435556    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.435628    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:46.935594684 +0000 UTC m=+2.977779351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435772    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-node-log\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.435941    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-kubelet\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436038    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysconfig\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed399293-2342-4edf-bf8b-2b333a686a3e-ovnkube-script-lib\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436154    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysctl-d\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436214    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-kubelet\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436260    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-run-netns\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436319    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-systemd\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436367    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5796db27-474f-4e9e-9b0a-a3a219c600b4-tmp-dir\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436380    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436463    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-slash\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.436904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-systemd-units\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436504    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-systemd-units\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436516    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-ovn\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436561    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5796db27-474f-4e9e-9b0a-a3a219c600b4-hosts-file\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-var-lib-kubelet\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436600    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-system-cni-dir\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-host\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436647    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-modprobe-d\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436655    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-host\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-etc-kubernetes\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436705    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-run-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436757    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-etc-kubernetes\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436771    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-var-lib-kubelet\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436819    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436831    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-sys\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436836    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-systemd\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.437800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436890    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-conf-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436892    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-socket-dir-parent\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436838    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-dbus\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436890    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-hostroot\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436966    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-cni-dir\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.436971    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-k8s-cni-cncf-io\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437012    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-sysctl-conf\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437072    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-netns\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437096    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437098    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cnibin\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437149    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-host-slash\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437161    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-etc-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437233    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2cd6863-1b67-4db0-b559-6d957315a9f9-lib-modules\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437207    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-var-lib-cni-multus\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.437689    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437722    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53f61516-48cb-43e6-95e1-f0b62350a522-konnectivity-ca\") pod \"konnectivity-agent-6zmpz\" (UID: \"53f61516-48cb-43e6-95e1-f0b62350a522\") " pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437894    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.438572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437943    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-kubelet-config\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.437980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvpsr\" (UniqueName: \"kubernetes.io/projected/be3ebbcc-0f5c-4d38-bde8-008358c5b341-kube-api-access-gvpsr\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.438005    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-host-cni-bin\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.438005    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed399293-2342-4edf-bf8b-2b333a686a3e-var-lib-openvswitch\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.438172    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be3ebbcc-0f5c-4d38-bde8-008358c5b341-multus-daemon-config\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.438237    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be3ebbcc-0f5c-4d38-bde8-008358c5b341-host-run-multus-certs\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.438442    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2f91405-83ae-4cc0-a508-d93ff9d99c56-cni-binary-copy\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.439372 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.439048    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:19:46.939016054 +0000 UTC m=+2.981200719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:46.440104 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.440053    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53f61516-48cb-43e6-95e1-f0b62350a522-agent-certs\") pod \"konnectivity-agent-6zmpz\" (UID: \"53f61516-48cb-43e6-95e1-f0b62350a522\") " pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.440180 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.440127    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2cd6863-1b67-4db0-b559-6d957315a9f9-tmp\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.440842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.440749    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2cd6863-1b67-4db0-b559-6d957315a9f9-etc-tuned\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.447827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.447797    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed399293-2342-4edf-bf8b-2b333a686a3e-ovn-node-metrics-cert\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.454211 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.454186    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fd4m\" (UniqueName: \"kubernetes.io/projected/b2f91405-83ae-4cc0-a508-d93ff9d99c56-kube-api-access-2fd4m\") pod \"multus-additional-cni-plugins-b5ppb\" (UID: \"b2f91405-83ae-4cc0-a508-d93ff9d99c56\") " pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.454665 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.454641    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vsjx\" (UniqueName: \"kubernetes.io/projected/ed399293-2342-4edf-bf8b-2b333a686a3e-kube-api-access-9vsjx\") pod \"ovnkube-node-hrq66\" (UID: \"ed399293-2342-4edf-bf8b-2b333a686a3e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.456696 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.456658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2r85\" (UniqueName: \"kubernetes.io/projected/8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb-kube-api-access-t2r85\") pod \"iptables-alerter-bjx6v\" (UID: \"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb\") " pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.462220 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.462102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal" event={"ID":"fa31c6dd93bd9d145e38cc39ff6f3247","Type":"ContainerStarted","Data":"74825a974b8ba1f0c11029b696e11de09665750d7e75a6f03555649bfb77d3be"}
Jul 08 03:19:46.463420 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.463391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal" event={"ID":"bd44f3e11daee88facac900a13ae9553","Type":"ContainerStarted","Data":"78606b74b7b009cd1fa8c34ec717a878458647eae8101959f2f0aabb2b88c84b"}
Jul 08 03:19:46.463934 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.463913    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvpsr\" (UniqueName: \"kubernetes.io/projected/be3ebbcc-0f5c-4d38-bde8-008358c5b341-kube-api-access-gvpsr\") pod \"multus-q5l55\" (UID: \"be3ebbcc-0f5c-4d38-bde8-008358c5b341\") " pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.464048 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.463961    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndf8m\" (UniqueName: \"kubernetes.io/projected/5796db27-474f-4e9e-9b0a-a3a219c600b4-kube-api-access-ndf8m\") pod \"node-resolver-sz6hp\" (UID: \"5796db27-474f-4e9e-9b0a-a3a219c600b4\") " pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.464048 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.463993    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccdhc\" (UniqueName: \"kubernetes.io/projected/f2cd6863-1b67-4db0-b559-6d957315a9f9-kube-api-access-ccdhc\") pod \"tuned-5v465\" (UID: \"f2cd6863-1b67-4db0-b559-6d957315a9f9\") " pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.465483 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.465455    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b78k\" (UniqueName: \"kubernetes.io/projected/b289c98c-c888-4672-9b09-03e12ae66c52-kube-api-access-4b78k\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.538396 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-registration-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538396 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-serviceca\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.538584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-device-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-etc-selinux\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-sys-fs\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn6n6\" (UniqueName: \"kubernetes.io/projected/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-kube-api-access-dn6n6\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-host\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.538765 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:46.538765 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538765 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4kbw\" (UniqueName: \"kubernetes.io/projected/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-kube-api-access-p4kbw\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.538765 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-socket-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538792    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-socket-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.538911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.538844    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-registration-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.539112 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.539080    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-device-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.539199 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.539163    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-sys-fs\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.539199 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.539177    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-etc-selinux\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.539288 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.539208    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-host\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.539288 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.539239    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.539288 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.539278    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-serviceca\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.548523 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.548501    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:46.548523 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.548526    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:46.548706 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.548538    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:46.548706 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.548598    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:47.048580385 +0000 UTC m=+3.090765054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:46.551892 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.551868    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn6n6\" (UniqueName: \"kubernetes.io/projected/c58ae2f4-10f0-45d2-ae52-a90e0d4a0767-kube-api-access-dn6n6\") pod \"aws-ebs-csi-driver-node-qhvkq\" (UID: \"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.552004 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.551917    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4kbw\" (UniqueName: \"kubernetes.io/projected/1f3445b3-8eb2-4137-8a58-f1637be7d7e4-kube-api-access-p4kbw\") pod \"node-ca-qwssj\" (UID: \"1f3445b3-8eb2-4137-8a58-f1637be7d7e4\") " pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.628696 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.628651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:19:46.639381 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.639351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sz6hp"
Jul 08 03:19:46.639705 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:46.639683    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded399293_2342_4edf_bf8b_2b333a686a3e.slice/crio-792e2ef17aada36a691aff260ef73df08ec37eedb9eeb98f36b7d5645fddb3fe WatchSource:0}: Error finding container 792e2ef17aada36a691aff260ef73df08ec37eedb9eeb98f36b7d5645fddb3fe: Status 404 returned error can't find the container with id 792e2ef17aada36a691aff260ef73df08ec37eedb9eeb98f36b7d5645fddb3fe
Jul 08 03:19:46.649201 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.649181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5v465"
Jul 08 03:19:46.657372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.657355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b5ppb"
Jul 08 03:19:46.664190 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.664167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q5l55"
Jul 08 03:19:46.675060 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.675040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjx6v"
Jul 08 03:19:46.684034 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.684017    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:19:46.692999 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.692980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq"
Jul 08 03:19:46.697617 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.697598    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwssj"
Jul 08 03:19:46.941418 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.941331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:46.941418 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:46.941379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:46.941624 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.941500    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:46.941624 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.941573    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:19:47.941551732 +0000 UTC m=+3.983736382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:46.941624 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.941500    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:46.941750 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:46.941673    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:47.941652997 +0000 UTC m=+3.983837649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:47.142844 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.142800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:47.143037 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.142976    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:47.143037 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.143014    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:47.143037 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.143027    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:47.143187 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.143089    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:48.143069776 +0000 UTC m=+4.185254427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:47.187630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.187600    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:47.389865 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.389785    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 03:14:46 +0000 UTC" deadline="2028-02-03 18:47:44.216205339 +0000 UTC"
Jul 08 03:19:47.389865 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.389823    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13815h27m56.826387585s"
Jul 08 03:19:47.452613 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.452581    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:47.452777 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.452713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:47.466434 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.466402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"792e2ef17aada36a691aff260ef73df08ec37eedb9eeb98f36b7d5645fddb3fe"}
Jul 08 03:19:47.711199 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:47.710431    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3ebbcc_0f5c_4d38_bde8_008358c5b341.slice/crio-d172f2e0abec6a36d92de19b9976f9c59fbbe150bce60a43ce8e1c1d4d9b26cf WatchSource:0}: Error finding container d172f2e0abec6a36d92de19b9976f9c59fbbe150bce60a43ce8e1c1d4d9b26cf: Status 404 returned error can't find the container with id d172f2e0abec6a36d92de19b9976f9c59fbbe150bce60a43ce8e1c1d4d9b26cf
Jul 08 03:19:47.712200 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:47.712170    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2cd6863_1b67_4db0_b559_6d957315a9f9.slice/crio-5f54f2f1907d903aef351c98dfbdbc289988d288343d1e03f3e0481698a95c0c WatchSource:0}: Error finding container 5f54f2f1907d903aef351c98dfbdbc289988d288343d1e03f3e0481698a95c0c: Status 404 returned error can't find the container with id 5f54f2f1907d903aef351c98dfbdbc289988d288343d1e03f3e0481698a95c0c
Jul 08 03:19:47.715484 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:47.715460    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3445b3_8eb2_4137_8a58_f1637be7d7e4.slice/crio-e1d84be4a812dd83cb9c535fc3c330ed7c5dbb76d23965b6d444c78e3808b047 WatchSource:0}: Error finding container e1d84be4a812dd83cb9c535fc3c330ed7c5dbb76d23965b6d444c78e3808b047: Status 404 returned error can't find the container with id e1d84be4a812dd83cb9c535fc3c330ed7c5dbb76d23965b6d444c78e3808b047
Jul 08 03:19:47.717534 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:47.716221    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58ae2f4_10f0_45d2_ae52_a90e0d4a0767.slice/crio-af2fd91e58faf89dbe3d73cb234f403605b9be5c3dba76a351807ba42513b9fd WatchSource:0}: Error finding container af2fd91e58faf89dbe3d73cb234f403605b9be5c3dba76a351807ba42513b9fd: Status 404 returned error can't find the container with id af2fd91e58faf89dbe3d73cb234f403605b9be5c3dba76a351807ba42513b9fd
Jul 08 03:19:47.717534 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:19:47.716846    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5796db27_474f_4e9e_9b0a_a3a219c600b4.slice/crio-655102d0790369f8892c6a2b3760dcf14a9fa7a4c8a7c6f036fd833e3af68d1e WatchSource:0}: Error finding container 655102d0790369f8892c6a2b3760dcf14a9fa7a4c8a7c6f036fd833e3af68d1e: Status 404 returned error can't find the container with id 655102d0790369f8892c6a2b3760dcf14a9fa7a4c8a7c6f036fd833e3af68d1e
Jul 08 03:19:47.950652 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.950397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:47.950652 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:47.950623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:47.950878 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.950646    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:47.950878 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.950731    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:19:49.950709832 +0000 UTC m=+5.992894487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:47.950878 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.950766    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:47.950878 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:47.950817    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:49.950802513 +0000 UTC m=+5.992987177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:48.152477 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.152431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:48.152674 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:48.152622    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:48.152674 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:48.152642    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:48.152674 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:48.152655    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:48.152841 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:48.152716    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:50.152696265 +0000 UTC m=+6.194880932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:48.390974 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.390867    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 03:14:46 +0000 UTC" deadline="2028-02-07 05:22:05.657590711 +0000 UTC"
Jul 08 03:19:48.390974 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.390939    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13898h2m17.266657054s"
Jul 08 03:19:48.467295 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.467239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:48.467442 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.467316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:48.467514 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:48.467434    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:48.467514 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:48.467486    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:48.476986 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.476954    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5v465" event={"ID":"f2cd6863-1b67-4db0-b559-6d957315a9f9","Type":"ContainerStarted","Data":"5f54f2f1907d903aef351c98dfbdbc289988d288343d1e03f3e0481698a95c0c"}
Jul 08 03:19:48.480131 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.480099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q5l55" event={"ID":"be3ebbcc-0f5c-4d38-bde8-008358c5b341","Type":"ContainerStarted","Data":"d172f2e0abec6a36d92de19b9976f9c59fbbe150bce60a43ce8e1c1d4d9b26cf"}
Jul 08 03:19:48.483347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.483314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal" event={"ID":"bd44f3e11daee88facac900a13ae9553","Type":"ContainerStarted","Data":"440b7f216af003367b359b1e1b852ffa794e2db0470e7803dfe435a76b08012c"}
Jul 08 03:19:48.488301 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.488277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6zmpz" event={"ID":"53f61516-48cb-43e6-95e1-f0b62350a522","Type":"ContainerStarted","Data":"8d4722f334452db4f5038deb3c46260290bd3da2b1a965a2f10fe7791f9b3dbf"}
Jul 08 03:19:48.490259 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.490232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq" event={"ID":"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767","Type":"ContainerStarted","Data":"af2fd91e58faf89dbe3d73cb234f403605b9be5c3dba76a351807ba42513b9fd"}
Jul 08 03:19:48.494959 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.494902    2580 generic.go:356] "Generic (PLEG): container finished" podID="fa31c6dd93bd9d145e38cc39ff6f3247" containerID="f4f5ebe6d11ab5979a059c79c677e1e149e31bf326faa9866a57abd2229ebc1c" exitCode=0
Jul 08 03:19:48.495589 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.495564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal" event={"ID":"fa31c6dd93bd9d145e38cc39ff6f3247","Type":"ContainerDied","Data":"f4f5ebe6d11ab5979a059c79c677e1e149e31bf326faa9866a57abd2229ebc1c"}
Jul 08 03:19:48.500090 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.500020    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-190.ec2.internal" podStartSLOduration=2.5000045269999998 podStartE2EDuration="2.500004527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:19:48.499643434 +0000 UTC m=+4.541828107" watchObservedRunningTime="2026-07-08 03:19:48.500004527 +0000 UTC m=+4.542189201"
Jul 08 03:19:48.509256 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.509170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjx6v" event={"ID":"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb","Type":"ContainerStarted","Data":"f1ab88c1d1ba662d513cb63deb4aaa20fe354747844a6395a0d92cba7e7a75c0"}
Jul 08 03:19:48.515208 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.514315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerStarted","Data":"914da6d150241d2ad8ff4c53ad62e18a8e94f8b491d5e66f4da81e964a9afa9e"}
Jul 08 03:19:48.524587 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.524556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sz6hp" event={"ID":"5796db27-474f-4e9e-9b0a-a3a219c600b4","Type":"ContainerStarted","Data":"655102d0790369f8892c6a2b3760dcf14a9fa7a4c8a7c6f036fd833e3af68d1e"}
Jul 08 03:19:48.535004 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:48.534960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwssj" event={"ID":"1f3445b3-8eb2-4137-8a58-f1637be7d7e4","Type":"ContainerStarted","Data":"e1d84be4a812dd83cb9c535fc3c330ed7c5dbb76d23965b6d444c78e3808b047"}
Jul 08 03:19:49.452538 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:49.452506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:49.453208 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:49.452654    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:49.565640 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:49.565208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal" event={"ID":"fa31c6dd93bd9d145e38cc39ff6f3247","Type":"ContainerStarted","Data":"3c61ed3135f52302522cf1c4812c2e67f68d4e32b0915287dcdfc6477505a2f4"}
Jul 08 03:19:49.981301 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:49.981262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:49.981477 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:49.981325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:49.981477 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:49.981451    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:49.981610 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:49.981511    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:53.981492414 +0000 UTC m=+10.023677065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:49.981951 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:49.981929    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:49.982056 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:49.981988    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:19:53.981972273 +0000 UTC m=+10.024156923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:50.183160 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:50.183122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:50.183378 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:50.183312    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:50.183378 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:50.183330    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:50.183378 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:50.183343    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:50.183609 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:50.183400    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:54.183381493 +0000 UTC m=+10.225566159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:50.453243 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:50.453159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:50.453776 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:50.453302    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:50.453995 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:50.453955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:50.454108 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:50.454086    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:51.452510 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:51.452476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:51.452705 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:51.452614    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:52.453215 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:52.453167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:52.453705 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:52.453324    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:52.453705 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:52.453385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:52.453705 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:52.453496    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:53.452392 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:53.451804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:53.452392 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:53.451962    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:54.016788 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:54.016755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:54.017262 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:54.016810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:54.017262 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.016912    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:54.017262 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.016975    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:02.016956055 +0000 UTC m=+18.059140709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:54.017443 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.017411    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:54.017583 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.017466    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:20:02.017448398 +0000 UTC m=+18.059633052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:54.219416 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:54.219356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:54.219605 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.219585    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:54.219675 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.219611    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:54.219675 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.219625    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:54.219775 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.219684    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:02.219664114 +0000 UTC m=+18.261848771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:54.453443 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:54.453416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:54.453616 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.453517    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:54.453905 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:54.453875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:54.454045 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:54.454006    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:55.452236 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:55.452203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:55.452739 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:55.452330    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:56.456040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:56.456012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:56.456536 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:56.456024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:56.456536 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:56.456141    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:56.456536 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:56.456221    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:57.452503 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:57.452473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:57.452683 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:57.452599    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:19:58.453385 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:58.453348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:19:58.453815 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:58.453492    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:19:58.453885 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:58.453861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:19:58.453988 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:58.453967    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:19:59.452059 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:19:59.452021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:19:59.452249 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:19:59.452180    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:00.452506 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:00.452470    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:00.452506 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:00.452512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:00.453033 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:00.452615    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:00.453033 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:00.452737    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:01.451867 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:01.451831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:01.452063 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:01.451936    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:02.081745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:02.081709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:02.082221 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:02.081792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:02.082221 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.081916    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:02.082221 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.081999    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:20:18.081980037 +0000 UTC m=+34.124164706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:02.082443 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.082427    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:02.082498 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.082472    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:18.082459966 +0000 UTC m=+34.124644613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:02.283144 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:02.283103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:02.283323 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.283249    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:20:02.283323 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.283272    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:20:02.283323 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.283285    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:02.283467 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.283340    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:18.283326727 +0000 UTC m=+34.325511373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:02.454916 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:02.454834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:02.455084 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.454964    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:02.455084 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:02.454978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:02.455182 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:02.455079    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:03.452678 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:03.452641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:03.453061 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:03.452796    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:04.454700 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:04.454667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:04.455708 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:04.454816    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:04.455708 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:04.454875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:04.455708 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:04.455003    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:05.452508 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:05.452475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:05.452658 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:05.452587    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:06.452281 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:06.452246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:06.452815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:06.452246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:06.452815 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:06.452391    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:06.452815 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:06.452454    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:07.452700 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.452562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:07.453199 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:07.452770    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:07.607285 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.607251    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2f91405-83ae-4cc0-a508-d93ff9d99c56" containerID="01fbf2c2ed817733a3074315f52d75f27f87ad1f44e63725b5ef9215823169c6" exitCode=0
Jul 08 03:20:07.607381 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.607361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerDied","Data":"01fbf2c2ed817733a3074315f52d75f27f87ad1f44e63725b5ef9215823169c6"}
Jul 08 03:20:07.609161 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.609132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sz6hp" event={"ID":"5796db27-474f-4e9e-9b0a-a3a219c600b4","Type":"ContainerStarted","Data":"56f5bcc329424d91a832f5a0e0d04176bd184e9255d9f80f65704714e9178808"}
Jul 08 03:20:07.610933 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.610904    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwssj" event={"ID":"1f3445b3-8eb2-4137-8a58-f1637be7d7e4","Type":"ContainerStarted","Data":"a3cbb0098e5362af27da3e5bf4b60a91160afcf8934c209ed3b62419c69ce955"}
Jul 08 03:20:07.612786 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.612764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5v465" event={"ID":"f2cd6863-1b67-4db0-b559-6d957315a9f9","Type":"ContainerStarted","Data":"a1850d195e6f4ad13e7ab5b89338c2476cbf08ad84681d866b03a0a7cc01c384"}
Jul 08 03:20:07.614671 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.614557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q5l55" event={"ID":"be3ebbcc-0f5c-4d38-bde8-008358c5b341","Type":"ContainerStarted","Data":"6540af3a070f51cd0ed5480b7ba57f95c0cf0dc3690a0c5d4f309a2b5245492d"}
Jul 08 03:20:07.616591 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.616572    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:20:07.616927 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.616902    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed399293-2342-4edf-bf8b-2b333a686a3e" containerID="808877548a64c1b0991b36bfbbc597fcd1f075119bcc747ee488473f9a22af46" exitCode=1
Jul 08 03:20:07.617007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.616981    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"7fc07c53708e531259db245e014e986fd624eb0ca73c462f1e8d7f408215afd0"}
Jul 08 03:20:07.617057 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.617035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerDied","Data":"808877548a64c1b0991b36bfbbc597fcd1f075119bcc747ee488473f9a22af46"}
Jul 08 03:20:07.617109 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.617074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"e83530ce7944916814271f4fc991641208fed9fb697fbd25970bb107c63faed1"}
Jul 08 03:20:07.618474 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.618453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6zmpz" event={"ID":"53f61516-48cb-43e6-95e1-f0b62350a522","Type":"ContainerStarted","Data":"ef02cb763bd0a9463c4ecc7de08705840c2881163d8b65d4dd1a4e01fcf1a013"}
Jul 08 03:20:07.620038 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.619990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq" event={"ID":"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767","Type":"ContainerStarted","Data":"f66336f22a49e45b9ae74b6d4b2930948e8255ca764248a8b1f35e2090c484a5"}
Jul 08 03:20:07.629718 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.629682    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-190.ec2.internal" podStartSLOduration=21.629671124 podStartE2EDuration="21.629671124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:19:49.583522977 +0000 UTC m=+5.625707642" watchObservedRunningTime="2026-07-08 03:20:07.629671124 +0000 UTC m=+23.671855794"
Jul 08 03:20:07.660651 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.660600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5v465" podStartSLOduration=3.227053013 podStartE2EDuration="22.660585195s" totalImagesPullingTime="19.433532182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:07.645291116 +0000 UTC m=+23.687475784" watchObservedRunningTime="2026-07-08 03:20:07.660585195 +0000 UTC m=+23.702769864"
Jul 08 03:20:07.679370 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.679329    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sz6hp" podStartSLOduration=3.251140605 podStartE2EDuration="22.67931526s" totalImagesPullingTime="19.428174655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:07.679294855 +0000 UTC m=+23.721479525" watchObservedRunningTime="2026-07-08 03:20:07.67931526 +0000 UTC m=+23.721499929"
Jul 08 03:20:07.679505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.679409    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qwssj" podStartSLOduration=3.597327464 podStartE2EDuration="22.679403317s" totalImagesPullingTime="19.082075853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:07.660515595 +0000 UTC m=+23.702700264" watchObservedRunningTime="2026-07-08 03:20:07.679403317 +0000 UTC m=+23.721587988"
Jul 08 03:20:07.698668 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.698599    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q5l55" podStartSLOduration=3.241144697 podStartE2EDuration="22.698579654s" totalImagesPullingTime="19.457434957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:07.698286898 +0000 UTC m=+23.740471568" watchObservedRunningTime="2026-07-08 03:20:07.698579654 +0000 UTC m=+23.740764327"
Jul 08 03:20:07.715711 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:07.715663    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6zmpz" podStartSLOduration=3.6386023659999998 podStartE2EDuration="22.715641179s" totalImagesPullingTime="19.077038813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:07.71511958 +0000 UTC m=+23.757304251" watchObservedRunningTime="2026-07-08 03:20:07.715641179 +0000 UTC m=+23.757825848"
Jul 08 03:20:08.230344 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.230090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:20:08.230719 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.230703    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:20:08.452820 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.452788    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:08.453339 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.452826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:08.453339 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:08.452921    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:08.453339 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:08.453039    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:08.626109 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.625681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjx6v" event={"ID":"8b61d939-5fcc-4e8e-8996-a38a4fdfc7eb","Type":"ContainerStarted","Data":"3ce83c52abe415eac6c258855ab05bb9e0c19761abf495b786a5f5451868224a"}
Jul 08 03:20:08.629482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.629447    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:20:08.629978 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.629936    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"73424ca6a79583e23dd4226ec4c2e13300d7dd084a26e6c531a30870f991e01b"}
Jul 08 03:20:08.630092 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.630056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"a7bc18b1cf67db8d501c9a65a9b443c880daa925196d26b5677d8cb2e28c72a0"}
Jul 08 03:20:08.630149 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.630100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"49593d226da295f7f5d836c0c0d70a0c827378ffdd91962404492b5cf6772623"}
Jul 08 03:20:08.630629 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.630612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:20:08.630850 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.630831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6zmpz"
Jul 08 03:20:08.641256 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.641216    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bjx6v" podStartSLOduration=4.22189017 podStartE2EDuration="23.641201416s" totalImagesPullingTime="19.419311246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:08.640752369 +0000 UTC m=+24.682937038" watchObservedRunningTime="2026-07-08 03:20:08.641201416 +0000 UTC m=+24.683386085"
Jul 08 03:20:08.801632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:08.801608    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 03:20:09.390704 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:09.390596    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T03:20:08.801630219Z","UUID":"30dc777b-dee4-4ffe-8fdf-ade8def36604","Handler":null,"Name":"","Endpoint":""}
Jul 08 03:20:09.392747 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:09.392725    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 03:20:09.392867 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:09.392767    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 03:20:09.452359 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:09.452329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:09.452498 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:09.452458    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:09.634823 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:09.634787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq" event={"ID":"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767","Type":"ContainerStarted","Data":"3cdb7697c262c219f2581f2ce2c79b0248eae09ad13bc03f1ab97d3a56ced484"}
Jul 08 03:20:10.452220 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:10.452182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:10.452220 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:10.452216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:10.452448 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:10.452316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:10.452448 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:10.452424    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:10.639505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:10.639479    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:20:10.639906 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:10.639830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"91b5f2184d95415efb7a7cb29f9fee7531733dca832a4693d91199a3b5f0f3bf"}
Jul 08 03:20:10.641561 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:10.641537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq" event={"ID":"c58ae2f4-10f0-45d2-ae52-a90e0d4a0767","Type":"ContainerStarted","Data":"68599e811fb766ee562cf9005925b38c175b61e0eee3b8cbff61ba6047a94d60"}
Jul 08 03:20:11.452840 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:11.452626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:11.453041 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:11.452938    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:12.452364 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:12.452328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:12.452364 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:12.452360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:12.452914 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:12.452457    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:12.452914 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:12.452603    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:13.452229 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.451998    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:13.452346 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:13.452236    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:13.651167 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.651136    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:20:13.651816 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.651525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"4b91c73a3238837baba745a131794d93ec13874bf8f8f13ed4193bd40901d5b0"}
Jul 08 03:20:13.651921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.651895    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:20:13.652067 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.651988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:20:13.652067 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.652026    2580 scope.go:117] "RemoveContainer" containerID="808877548a64c1b0991b36bfbbc597fcd1f075119bcc747ee488473f9a22af46"
Jul 08 03:20:13.654706 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.653349    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2f91405-83ae-4cc0-a508-d93ff9d99c56" containerID="84e4280d515160d269a3c1f75be75be655d5932b2a6bac680e7f8a4c6d43fb56" exitCode=0
Jul 08 03:20:13.654706 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.653393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerDied","Data":"84e4280d515160d269a3c1f75be75be655d5932b2a6bac680e7f8a4c6d43fb56"}
Jul 08 03:20:13.668764 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.668747    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:20:13.678271 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:13.678225    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qhvkq" podStartSLOduration=11.652846066 podStartE2EDuration="28.678212746s" totalImagesPullingTime="17.02536668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:10.662426604 +0000 UTC m=+26.704611285" watchObservedRunningTime="2026-07-08 03:20:13.678212746 +0000 UTC m=+29.720397415"
Jul 08 03:20:14.452902 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.452876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:14.453139 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:14.452974    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:14.453139 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.453024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:14.453245 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:14.453131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:14.661496 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.661473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:20:14.661890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.661858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" event={"ID":"ed399293-2342-4edf-bf8b-2b333a686a3e","Type":"ContainerStarted","Data":"f1207e9127faef4f887a435cde7aff77f59cf55150fb12f9e2e7429d54e13eb1"}
Jul 08 03:20:14.662158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.662140    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:20:14.677964 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.677925    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:20:14.690890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.690844    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66" podStartSLOduration=9.125931403 podStartE2EDuration="29.690831948s" totalImagesPullingTime="20.564900545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:14.690008048 +0000 UTC m=+30.732192718" watchObservedRunningTime="2026-07-08 03:20:14.690831948 +0000 UTC m=+30.733016618"
Jul 08 03:20:14.849556 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.849524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txl57"]
Jul 08 03:20:14.849707 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.849635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:14.849749 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:14.849718    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:14.852472 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.852443    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dflvf"]
Jul 08 03:20:14.852610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.852546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:14.852668 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:14.852650    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:14.853158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.853125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8cxvd"]
Jul 08 03:20:14.853246 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:14.853237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:14.853358 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:14.853336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:15.666314 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:15.666278    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2f91405-83ae-4cc0-a508-d93ff9d99c56" containerID="96621890241de33f5417884657315d80e695def762901fd8f7bc5906eb6ad004" exitCode=0
Jul 08 03:20:15.666767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:15.666365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerDied","Data":"96621890241de33f5417884657315d80e695def762901fd8f7bc5906eb6ad004"}
Jul 08 03:20:16.456415 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:16.456192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:16.456566 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:16.456442    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:16.456840 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:16.456824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:16.456943 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:16.456924    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:16.457013 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:16.457002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:16.457102 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:16.457084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:16.670199 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:16.670173    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2f91405-83ae-4cc0-a508-d93ff9d99c56" containerID="1b1e18407bf20bdb900e4909eeb87bddcedad41cd1dc07678bf52a9db98f9827" exitCode=0
Jul 08 03:20:16.670732 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:16.670256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerDied","Data":"1b1e18407bf20bdb900e4909eeb87bddcedad41cd1dc07678bf52a9db98f9827"}
Jul 08 03:20:18.101724 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:18.101689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:18.101724 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:18.101729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:18.102207 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.101838    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:18.102207 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.101843    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:18.102207 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.101891    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:50.101875254 +0000 UTC m=+66.144059901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:18.102207 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.101910    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret podName:122f7785-2bbc-41e8-99fe-cbc1d4c71b7b nodeName:}" failed. No retries permitted until 2026-07-08 03:20:50.101900112 +0000 UTC m=+66.144084763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret") pod "global-pull-secret-syncer-txl57" (UID: "122f7785-2bbc-41e8-99fe-cbc1d4c71b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:18.302999 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:18.302971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:18.303152 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.303089    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:20:18.303152 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.303103    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:20:18.303152 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.303112    2580 projected.go:196] Error preparing data for projected volume kube-api-access-qzgj9 for pod openshift-network-diagnostics/network-check-target-dflvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:18.303260 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.303158    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9 podName:1c577e13-7f68-486c-b4a4-57ab25da3939 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:50.303145261 +0000 UTC m=+66.345329908 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzgj9" (UniqueName: "kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9") pod "network-check-target-dflvf" (UID: "1c577e13-7f68-486c-b4a4-57ab25da3939") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:18.452648 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:18.452609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:18.452828 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:18.452612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:18.452892 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.452728    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txl57" podUID="122f7785-2bbc-41e8-99fe-cbc1d4c71b7b"
Jul 08 03:20:18.452892 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.452861    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:20:18.452892 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:18.452861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:18.453075 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:18.452932    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflvf" podUID="1c577e13-7f68-486c-b4a4-57ab25da3939"
Jul 08 03:20:20.210510 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.210482    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-190.ec2.internal" event="NodeReady"
Jul 08 03:20:20.210992 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.210604    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 03:20:20.245990 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.245210    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"]
Jul 08 03:20:20.247226 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.247209    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66b449c4f7-sp8qb"]
Jul 08 03:20:20.247370 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.247347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:20.249815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.249479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.249815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.249505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gk4k4\""
Jul 08 03:20:20.249815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.249550    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 08 03:20:20.249815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.249599    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 08 03:20:20.251672 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.251649    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 03:20:20.252148 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.252112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5t5kv\""
Jul 08 03:20:20.252376 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.252362    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 03:20:20.252724 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.252695    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 03:20:20.257293 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.257113    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 03:20:20.259402 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.259381    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"]
Jul 08 03:20:20.262212 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.262189    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r8rwb"]
Jul 08 03:20:20.266272 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.264723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.266372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.266303    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4vvts"]
Jul 08 03:20:20.267352 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.267331    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gjrkj\""
Jul 08 03:20:20.267352 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.267540    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 03:20:20.267937 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.267715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 03:20:20.268527 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.268512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b449c4f7-sp8qb"]
Jul 08 03:20:20.268625 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.268584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.270410 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.270395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v57gq\""
Jul 08 03:20:20.271225 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.271205    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 03:20:20.271315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.271229    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 03:20:20.271315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.271216    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 03:20:20.274304 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.274286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8rwb"]
Jul 08 03:20:20.277697 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.277674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4vvts"]
Jul 08 03:20:20.417756 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.417725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/919205f6-7f4b-434c-9564-312f471eaf83-ca-trust-extracted\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.417756 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.417762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418044 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.417805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-bound-sa-token\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418044 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.417840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3458bec-0dc7-42ff-b199-84cbafc31675-tmp-dir\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.418044 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.417981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-installation-pull-secrets\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418198 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1d4333f-55cc-414a-a948-48d70aef566e-cert\") pod \"ingress-canary-4vvts\" (UID: \"c1d4333f-55cc-414a-a948-48d70aef566e\") " pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.418198 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5e7b6863-5540-4e0c-a79b-a725921fcd4e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:20.418198 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f77sm\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-kube-api-access-f77sm\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418198 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3458bec-0dc7-42ff-b199-84cbafc31675-config-volume\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-image-registry-private-configuration\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-registry-certificates\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-trusted-ca\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5tbl\" (UniqueName: \"kubernetes.io/projected/a3458bec-0dc7-42ff-b199-84cbafc31675-kube-api-access-m5tbl\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:20.418395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.418369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nppks\" (UniqueName: \"kubernetes.io/projected/c1d4333f-55cc-414a-a948-48d70aef566e-kube-api-access-nppks\") pod \"ingress-canary-4vvts\" (UID: \"c1d4333f-55cc-414a-a948-48d70aef566e\") " pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.452028 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.451993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:20.452193 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.452047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:20.452193 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.452059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:20.454771 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.454741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 03:20:20.454896 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.454763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 03:20:20.454896 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.454880    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 03:20:20.455055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.455042    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zb8ld\""
Jul 08 03:20:20.455130 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.455116    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 03:20:20.455130 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.455122    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2m54\""
Jul 08 03:20:20.519416 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5tbl\" (UniqueName: \"kubernetes.io/projected/a3458bec-0dc7-42ff-b199-84cbafc31675-kube-api-access-m5tbl\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nppks\" (UniqueName: \"kubernetes.io/projected/c1d4333f-55cc-414a-a948-48d70aef566e-kube-api-access-nppks\") pod \"ingress-canary-4vvts\" (UID: \"c1d4333f-55cc-414a-a948-48d70aef566e\") " pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/919205f6-7f4b-434c-9564-312f471eaf83-ca-trust-extracted\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-bound-sa-token\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3458bec-0dc7-42ff-b199-84cbafc31675-tmp-dir\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.519566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-installation-pull-secrets\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.519568    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1d4333f-55cc-414a-a948-48d70aef566e-cert\") pod \"ingress-canary-4vvts\" (UID: \"c1d4333f-55cc-414a-a948-48d70aef566e\") " pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5e7b6863-5540-4e0c-a79b-a725921fcd4e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.519637    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:21.019614686 +0000 UTC m=+37.061799350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f77sm\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-kube-api-access-f77sm\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.519692    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.519710    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3458bec-0dc7-42ff-b199-84cbafc31675-config-volume\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.519775    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:21.019757942 +0000 UTC m=+37.061942592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-image-registry-private-configuration\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-registry-certificates\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519854    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/919205f6-7f4b-434c-9564-312f471eaf83-ca-trust-extracted\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-trusted-ca\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.519900 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.519896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.520513 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.520007    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:20.520513 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:20.520047    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:21.020032868 +0000 UTC m=+37.062217515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:20.520513 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.520110    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3458bec-0dc7-42ff-b199-84cbafc31675-tmp-dir\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.520513 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.520361    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3458bec-0dc7-42ff-b199-84cbafc31675-config-volume\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.520934 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.520719    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-registry-certificates\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.520934 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.520763    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5e7b6863-5540-4e0c-a79b-a725921fcd4e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:20.521056 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.521011    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-trusted-ca\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.523800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.523781    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-installation-pull-secrets\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.523889 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.523804    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1d4333f-55cc-414a-a948-48d70aef566e-cert\") pod \"ingress-canary-4vvts\" (UID: \"c1d4333f-55cc-414a-a948-48d70aef566e\") " pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.523889 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.523809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-image-registry-private-configuration\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.529041 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.529023    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5tbl\" (UniqueName: \"kubernetes.io/projected/a3458bec-0dc7-42ff-b199-84cbafc31675-kube-api-access-m5tbl\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:20.529342 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.529326    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f77sm\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-kube-api-access-f77sm\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.530476 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.530457    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-bound-sa-token\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:20.530981 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.530945    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nppks\" (UniqueName: \"kubernetes.io/projected/c1d4333f-55cc-414a-a948-48d70aef566e-kube-api-access-nppks\") pod \"ingress-canary-4vvts\" (UID: \"c1d4333f-55cc-414a-a948-48d70aef566e\") " pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.589056 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.589023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4vvts"
Jul 08 03:20:20.729242 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:20.729087    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4vvts"]
Jul 08 03:20:21.024496 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:21.024411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:21.024496 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:21.024474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:21.024521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024589    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024626    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024626    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024636    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024656    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:22.02464039 +0000 UTC m=+38.066825037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024682    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:22.024664991 +0000 UTC m=+38.066849658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:21.024720 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:21.024701    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:22.024691388 +0000 UTC m=+38.066876043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:22.032188 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:22.032147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:22.032266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:22.032297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032327    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032353    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032425    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032425    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032429    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:24.032406658 +0000 UTC m=+40.074591322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032486    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:24.032473862 +0000 UTC m=+40.074658509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:22.032628 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:22.032504    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:24.03249701 +0000 UTC m=+40.074681657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:22.286628 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:20:22.286560    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d4333f_55cc_414a_a948_48d70aef566e.slice/crio-03f2077fec1e8b5bb45f88bb4d7734b66178d0051430724fe39fd7d447f10cd7 WatchSource:0}: Error finding container 03f2077fec1e8b5bb45f88bb4d7734b66178d0051430724fe39fd7d447f10cd7: Status 404 returned error can't find the container with id 03f2077fec1e8b5bb45f88bb4d7734b66178d0051430724fe39fd7d447f10cd7
Jul 08 03:20:22.685441 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:22.685411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4vvts" event={"ID":"c1d4333f-55cc-414a-a948-48d70aef566e","Type":"ContainerStarted","Data":"03f2077fec1e8b5bb45f88bb4d7734b66178d0051430724fe39fd7d447f10cd7"}
Jul 08 03:20:23.690782 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:23.690745    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2f91405-83ae-4cc0-a508-d93ff9d99c56" containerID="6b92034510bd1138f0764e3cae43a431f3d06357588ad5cc5f905f0f3eaf2d68" exitCode=0
Jul 08 03:20:23.691171 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:23.690790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerDied","Data":"6b92034510bd1138f0764e3cae43a431f3d06357588ad5cc5f905f0f3eaf2d68"}
Jul 08 03:20:24.048782 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:24.048747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:24.048924 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:24.048786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:24.048924 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:24.048836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:24.048924 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.048901    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:24.049053 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.048982    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:28.048943476 +0000 UTC m=+44.091128147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:24.049053 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.048993    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:24.049053 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.049010    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:24.049053 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.049023    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:24.049053 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.049048    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:28.049033962 +0000 UTC m=+44.091218610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:24.049211 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:24.049069    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:28.049057805 +0000 UTC m=+44.091242456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:24.697130 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:24.696885    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2f91405-83ae-4cc0-a508-d93ff9d99c56" containerID="9314ac7b9d4923c25b83d9d6917ab2f6e42b547a6dedfafd5fe74866529f3712" exitCode=0
Jul 08 03:20:24.697130 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:24.696981    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerDied","Data":"9314ac7b9d4923c25b83d9d6917ab2f6e42b547a6dedfafd5fe74866529f3712"}
Jul 08 03:20:25.426085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.426055    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:25.512450 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.512408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:25.701124 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.701089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4vvts" event={"ID":"c1d4333f-55cc-414a-a948-48d70aef566e","Type":"ContainerStarted","Data":"c09edb000efc8f47ec3874e238aed0ac20729b8ec75ae8420271bf813e0ae617"}
Jul 08 03:20:25.704196 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.704167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" event={"ID":"b2f91405-83ae-4cc0-a508-d93ff9d99c56","Type":"ContainerStarted","Data":"9c29879a77d2214d2f611222cb864821f70870e2c8c0a410195d9bd0f0130182"}
Jul 08 03:20:25.716935 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.716896    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4vvts" podStartSLOduration=2.713756197 podStartE2EDuration="5.716883043s" totalImagesPullingTime="3.003126846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:25.716355417 +0000 UTC m=+41.758540089" watchObservedRunningTime="2026-07-08 03:20:25.716883043 +0000 UTC m=+41.759067708"
Jul 08 03:20:25.738611 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.738567    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b5ppb" podStartSLOduration=7.727653564 podStartE2EDuration="40.738553654s" totalImagesPullingTime="33.01090009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:25.737301684 +0000 UTC m=+41.779486352" watchObservedRunningTime="2026-07-08 03:20:25.738553654 +0000 UTC m=+41.780738323"
Jul 08 03:20:25.859581 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.859537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:25.928459 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:25.928430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:26.613082 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:26.613044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:26.671037 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:26.671002    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:27.834048 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:27.834013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:27.889148 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:27.889111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:28.080333 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:28.080294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:28.080518 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:28.080337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:28.080518 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:28.080386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:28.080518 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080457    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:28.080518 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080493    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:28.080518 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080499    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:28.080518 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080512    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:28.080727 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080538    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:36.080516359 +0000 UTC m=+52.122701049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:28.080727 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080557    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:36.080545606 +0000 UTC m=+52.122730274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:28.080727 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:28.080573    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:36.080564876 +0000 UTC m=+52.122749523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:32.275637 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:32.275594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:32.332604 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:32.332578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:33.294341 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:33.294306    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:33.356156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:33.356129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:36.142986 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:36.142930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:36.143029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:36.143050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143079    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143098    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143156    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:52.143140572 +0000 UTC m=+68.185325219 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143157    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143155    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:52.143195056 +0000 UTC m=+68.185379703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:36.143393 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:36.143281    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:52.143267387 +0000 UTC m=+68.185452039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:36.225379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:36.225348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:37.172898 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:37.172864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:37.225524 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:37.225493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:46.680704 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:46.680677    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrq66"
Jul 08 03:20:46.832168 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:46.832132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:46.933438 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:46.933348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:49.089035 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:49.088991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:49.190216 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:49.190181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:50.145054 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.145016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:50.145432 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.145073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:20:50.147560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.147541    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 03:20:50.147616 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.147546    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 03:20:50.155434 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:50.155414    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 03:20:50.155510 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:50.155478    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:21:54.155459264 +0000 UTC m=+130.197643911 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : secret "metrics-daemon-secret" not found
Jul 08 03:20:50.159093 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.159076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/122f7785-2bbc-41e8-99fe-cbc1d4c71b7b-original-pull-secret\") pod \"global-pull-secret-syncer-txl57\" (UID: \"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b\") " pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:50.171147 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.171121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txl57"
Jul 08 03:20:50.288679 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.288650    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txl57"]
Jul 08 03:20:50.292336 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:20:50.292309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122f7785_2bbc_41e8_99fe_cbc1d4c71b7b.slice/crio-3edba9c8b2fe8a32ea73f884098a613d40d7b8eb391dc83d20a9b5886a8983ef WatchSource:0}: Error finding container 3edba9c8b2fe8a32ea73f884098a613d40d7b8eb391dc83d20a9b5886a8983ef: Status 404 returned error can't find the container with id 3edba9c8b2fe8a32ea73f884098a613d40d7b8eb391dc83d20a9b5886a8983ef
Jul 08 03:20:50.346733 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.346704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:50.349471 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.349453    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 03:20:50.359487 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.359462    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 03:20:50.370612 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.370593    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzgj9\" (UniqueName: \"kubernetes.io/projected/1c577e13-7f68-486c-b4a4-57ab25da3939-kube-api-access-qzgj9\") pod \"network-check-target-dflvf\" (UID: \"1c577e13-7f68-486c-b4a4-57ab25da3939\") " pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:50.466993 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.466926    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2m54\""
Jul 08 03:20:50.475575 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.475555    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:50.593879 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.593850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dflvf"]
Jul 08 03:20:50.596992 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:20:50.596967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c577e13_7f68_486c_b4a4_57ab25da3939.slice/crio-b7d4243b4b164906fa27b5584d6a7f7cf692720dbc1ab197f9a047adbe209e15 WatchSource:0}: Error finding container b7d4243b4b164906fa27b5584d6a7f7cf692720dbc1ab197f9a047adbe209e15: Status 404 returned error can't find the container with id b7d4243b4b164906fa27b5584d6a7f7cf692720dbc1ab197f9a047adbe209e15
Jul 08 03:20:50.713082 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.713056    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:50.761419 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.761353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dflvf" event={"ID":"1c577e13-7f68-486c-b4a4-57ab25da3939","Type":"ContainerStarted","Data":"b7d4243b4b164906fa27b5584d6a7f7cf692720dbc1ab197f9a047adbe209e15"}
Jul 08 03:20:50.762350 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:50.762331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txl57" event={"ID":"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b","Type":"ContainerStarted","Data":"3edba9c8b2fe8a32ea73f884098a613d40d7b8eb391dc83d20a9b5886a8983ef"}
Jul 08 03:20:51.819912 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:51.819861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:51.881289 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:51.881262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:52.161888 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:52.161795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:20:52.161888 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:52.161840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:20:52.161888 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:52.161868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.161969    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.162001    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.162035    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:21:24.162017867 +0000 UTC m=+100.204202514 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.162057    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:21:24.162040941 +0000 UTC m=+100.204225588 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.162001    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.162076    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:20:52.162113 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:20:52.162107    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:21:24.162097634 +0000 UTC m=+100.204282287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:20:52.379076 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:52.379045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:52.434455 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:52.434372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:20:56.780742 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:56.780691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dflvf" event={"ID":"1c577e13-7f68-486c-b4a4-57ab25da3939","Type":"ContainerStarted","Data":"efd4fbf2d64701616410cdf020df15a89d125437df69e8d6928e155f8be4a784"}
Jul 08 03:20:56.781170 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:56.780760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:20:56.782168 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:56.782144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txl57" event={"ID":"122f7785-2bbc-41e8-99fe-cbc1d4c71b7b","Type":"ContainerStarted","Data":"0b136522e204a7f7880bda4d74d410e7671423b3fd730c1aaf0b600e172b6fc9"}
Jul 08 03:20:56.796147 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:20:56.796094    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dflvf" podStartSLOduration=66.48797057 podStartE2EDuration="1m11.796082085s" totalImagesPullingTime="5.308111515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:56.795808381 +0000 UTC m=+72.837993051" watchObservedRunningTime="2026-07-08 03:20:56.796082085 +0000 UTC m=+72.838266748"
Jul 08 03:21:11.340024 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:11.339889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:21:11.402238 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:11.402203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:21:24.184908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:24.184850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:24.184914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:24.184972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185028    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185041    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185104    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:22:28.18508725 +0000 UTC m=+164.227271897 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185120    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:22:28.185112992 +0000 UTC m=+164.227297640 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185149    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185170    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:21:24.185407 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:24.185223    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:22:28.185209007 +0000 UTC m=+164.227393655 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:21:27.787024 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:27.786993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dflvf"
Jul 08 03:21:27.804318 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:27.804270    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-txl57" podStartSLOduration=97.179844273 podStartE2EDuration="1m42.804255362s" totalImagesPullingTime="5.624411089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:56.809029214 +0000 UTC m=+72.851213882" watchObservedRunningTime="2026-07-08 03:21:27.804255362 +0000 UTC m=+103.846440035"
Jul 08 03:21:54.189618 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:21:54.189577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:21:54.190112 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:54.189715    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 03:21:54.190112 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:21:54.189786    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:23:56.189771535 +0000 UTC m=+252.231956181 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : secret "metrics-daemon-secret" not found
Jul 08 03:22:23.262133 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:23.262075    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z" podUID="5e7b6863-5540-4e0c-a79b-a725921fcd4e"
Jul 08 03:22:23.271236 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:23.271207    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" podUID="919205f6-7f4b-434c-9564-312f471eaf83"
Jul 08 03:22:23.283350 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:23.283331    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-r8rwb" podUID="a3458bec-0dc7-42ff-b199-84cbafc31675"
Jul 08 03:22:23.475627 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:23.475580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:22:23.988901 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:23.988875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:22:23.988901 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:23.988887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:22:23.989117 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:23.988877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:22:28.222256 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:28.222220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:28.222289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:28.222325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222385    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222407    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b449c4f7-sp8qb: secret "image-registry-tls" not found
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222411    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222391    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222471    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls podName:919205f6-7f4b-434c-9564-312f471eaf83 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:30.222454373 +0000 UTC m=+286.264639020 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls") pod "image-registry-66b449c4f7-sp8qb" (UID: "919205f6-7f4b-434c-9564-312f471eaf83") : secret "image-registry-tls" not found
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222485    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:24:30.222478962 +0000 UTC m=+286.264663610 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:22:28.222657 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:22:28.222496    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls podName:a3458bec-0dc7-42ff-b199-84cbafc31675 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:30.222490724 +0000 UTC m=+286.264675370 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls") pod "dns-default-r8rwb" (UID: "a3458bec-0dc7-42ff-b199-84cbafc31675") : secret "dns-default-metrics-tls" not found
Jul 08 03:22:36.452126 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:22:36.452041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:23:34.182914 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:23:34.182884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:23:56.240181 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:23:56.240137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:23:56.240636 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:23:56.240283    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 03:23:56.240636 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:23:56.240349    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs podName:b289c98c-c888-4672-9b09-03e12ae66c52 nodeName:}" failed. No retries permitted until 2026-07-08 03:25:58.240333769 +0000 UTC m=+374.282518416 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs") pod "network-metrics-daemon-8cxvd" (UID: "b289c98c-c888-4672-9b09-03e12ae66c52") : secret "metrics-daemon-secret" not found
Jul 08 03:24:13.306581 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.306496    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"]
Jul 08 03:24:13.308664 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.308639    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"]
Jul 08 03:24:13.308777 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.308654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"
Jul 08 03:24:13.310253 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.310237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.311049 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.311028    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vslj4\""
Jul 08 03:24:13.311687 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.311668    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.311772 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.311668    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.312275 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.312223    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.312599 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.312581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 08 03:24:13.312999 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.312983    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"]
Jul 08 03:24:13.314055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.314020    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.314156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.314071    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 08 03:24:13.314156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.314071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4jd49\""
Jul 08 03:24:13.314584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.314569    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.316757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.316742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.317031 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.317010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 08 03:24:13.317348 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.317335    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rrch6\""
Jul 08 03:24:13.317588 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.317570    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.321749 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.321728    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"]
Jul 08 03:24:13.325807 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.325789    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"]
Jul 08 03:24:13.330203 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.330181    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"]
Jul 08 03:24:13.351612 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.351587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ms6s\" (UniqueName: \"kubernetes.io/projected/68111e66-8ce8-4eec-b76b-363147c018eb-kube-api-access-6ms6s\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.351699 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.351614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/68111e66-8ce8-4eec-b76b-363147c018eb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.351699 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.351636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8m6s\" (UniqueName: \"kubernetes.io/projected/88fab518-b439-4c75-85bf-52139e1bd0f5-kube-api-access-b8m6s\") pod \"volume-data-source-validator-5d456fdbcd-mqq6g\" (UID: \"88fab518-b439-4c75-85bf-52139e1bd0f5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"
Jul 08 03:24:13.351779 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.351701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.351779 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.351744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68111e66-8ce8-4eec-b76b-363147c018eb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.351779 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.351764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52vbw\" (UniqueName: \"kubernetes.io/projected/2e8f6056-9a9e-4532-8224-6189831b56af-kube-api-access-52vbw\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.401220 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.401199    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"]
Jul 08 03:24:13.403341 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.403322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"
Jul 08 03:24:13.405570 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.405557    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jhvt4\""
Jul 08 03:24:13.407679 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.407660    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"]
Jul 08 03:24:13.409452 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.409442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.412131 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.412112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 08 03:24:13.413531 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.413506    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.415925 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.414485    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.415925 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.414714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zmtd9\""
Jul 08 03:24:13.416844 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.416821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"]
Jul 08 03:24:13.417549 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.417529    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 08 03:24:13.423608 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.423587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"]
Jul 08 03:24:13.452890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.452862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khdjr\" (UniqueName: \"kubernetes.io/projected/78e7b8a1-4722-4273-b358-1dc16d59d957-kube-api-access-khdjr\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.452890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.452891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt64c\" (UniqueName: \"kubernetes.io/projected/5c40feac-bd39-4fac-ac53-7c37fca4e846-kube-api-access-zt64c\") pod \"network-check-source-989f8b8dc-75bj4\" (UID: \"5c40feac-bd39-4fac-ac53-7c37fca4e846\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"
Jul 08 03:24:13.453055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.452918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ms6s\" (UniqueName: \"kubernetes.io/projected/68111e66-8ce8-4eec-b76b-363147c018eb-kube-api-access-6ms6s\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.453055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.452941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/68111e66-8ce8-4eec-b76b-363147c018eb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.453055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.452998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8m6s\" (UniqueName: \"kubernetes.io/projected/88fab518-b439-4c75-85bf-52139e1bd0f5-kube-api-access-b8m6s\") pod \"volume-data-source-validator-5d456fdbcd-mqq6g\" (UID: \"88fab518-b439-4c75-85bf-52139e1bd0f5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"
Jul 08 03:24:13.453055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.453038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/78e7b8a1-4722-4273-b358-1dc16d59d957-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.453219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.453062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.453219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.453091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68111e66-8ce8-4eec-b76b-363147c018eb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.453219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.453117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52vbw\" (UniqueName: \"kubernetes.io/projected/2e8f6056-9a9e-4532-8224-6189831b56af-kube-api-access-52vbw\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.453219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.453140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.453415 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:13.453272    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 03:24:13.453415 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:13.453339    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls podName:2e8f6056-9a9e-4532-8224-6189831b56af nodeName:}" failed. No retries permitted until 2026-07-08 03:24:13.953319588 +0000 UTC m=+269.995504236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-tw97b" (UID: "2e8f6056-9a9e-4532-8224-6189831b56af") : secret "samples-operator-tls" not found
Jul 08 03:24:13.453671 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.453648    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68111e66-8ce8-4eec-b76b-363147c018eb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.455362 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.455344    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/68111e66-8ce8-4eec-b76b-363147c018eb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.462921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.462899    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8m6s\" (UniqueName: \"kubernetes.io/projected/88fab518-b439-4c75-85bf-52139e1bd0f5-kube-api-access-b8m6s\") pod \"volume-data-source-validator-5d456fdbcd-mqq6g\" (UID: \"88fab518-b439-4c75-85bf-52139e1bd0f5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"
Jul 08 03:24:13.463252 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.463234    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52vbw\" (UniqueName: \"kubernetes.io/projected/2e8f6056-9a9e-4532-8224-6189831b56af-kube-api-access-52vbw\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.463324 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.463299    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ms6s\" (UniqueName: \"kubernetes.io/projected/68111e66-8ce8-4eec-b76b-363147c018eb-kube-api-access-6ms6s\") pod \"kube-storage-version-migrator-operator-846bf668f7-5lrfp\" (UID: \"68111e66-8ce8-4eec-b76b-363147c018eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.546594 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.546563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:13.553402 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.553380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/78e7b8a1-4722-4273-b358-1dc16d59d957-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.553473 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.553418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.553473 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.553452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khdjr\" (UniqueName: \"kubernetes.io/projected/78e7b8a1-4722-4273-b358-1dc16d59d957-kube-api-access-khdjr\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.553550 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:13.553523    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:13.553598 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:13.553576    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls podName:78e7b8a1-4722-4273-b358-1dc16d59d957 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:14.053560712 +0000 UTC m=+270.095745359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-gvdbh" (UID: "78e7b8a1-4722-4273-b358-1dc16d59d957") : secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:13.553598 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.553572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt64c\" (UniqueName: \"kubernetes.io/projected/5c40feac-bd39-4fac-ac53-7c37fca4e846-kube-api-access-zt64c\") pod \"network-check-source-989f8b8dc-75bj4\" (UID: \"5c40feac-bd39-4fac-ac53-7c37fca4e846\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"
Jul 08 03:24:13.554079 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.554062    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/78e7b8a1-4722-4273-b358-1dc16d59d957-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.563529 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.563490    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khdjr\" (UniqueName: \"kubernetes.io/projected/78e7b8a1-4722-4273-b358-1dc16d59d957-kube-api-access-khdjr\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:13.563633 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.563613    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt64c\" (UniqueName: \"kubernetes.io/projected/5c40feac-bd39-4fac-ac53-7c37fca4e846-kube-api-access-zt64c\") pod \"network-check-source-989f8b8dc-75bj4\" (UID: \"5c40feac-bd39-4fac-ac53-7c37fca4e846\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"
Jul 08 03:24:13.618731 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.618691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"
Jul 08 03:24:13.625146 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.625130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"
Jul 08 03:24:13.647555 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.647511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:13.713208 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.713178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"
Jul 08 03:24:13.748767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.748733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g"]
Jul 08 03:24:13.752345 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:13.752319    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88fab518_b439_4c75_85bf_52139e1bd0f5.slice/crio-fed67557e1f5791f2c3f6342d1817dc1f3fe788b3686832d1ffe4b19dff5949e WatchSource:0}: Error finding container fed67557e1f5791f2c3f6342d1817dc1f3fe788b3686832d1ffe4b19dff5949e: Status 404 returned error can't find the container with id fed67557e1f5791f2c3f6342d1817dc1f3fe788b3686832d1ffe4b19dff5949e
Jul 08 03:24:13.768256 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.768232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp"]
Jul 08 03:24:13.770498 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:13.770471    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68111e66_8ce8_4eec_b76b_363147c018eb.slice/crio-9d441837d7801d835640928ada2eddfe18f9bc5c5380f11c16817028564e4293 WatchSource:0}: Error finding container 9d441837d7801d835640928ada2eddfe18f9bc5c5380f11c16817028564e4293: Status 404 returned error can't find the container with id 9d441837d7801d835640928ada2eddfe18f9bc5c5380f11c16817028564e4293
Jul 08 03:24:13.840141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.840083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4"]
Jul 08 03:24:13.842752 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:13.842729    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c40feac_bd39_4fac_ac53_7c37fca4e846.slice/crio-843b1eea7c5be0d1c25b1d992fcad9af3871a8db82bd4048fcbe55f6849bb83a WatchSource:0}: Error finding container 843b1eea7c5be0d1c25b1d992fcad9af3871a8db82bd4048fcbe55f6849bb83a: Status 404 returned error can't find the container with id 843b1eea7c5be0d1c25b1d992fcad9af3871a8db82bd4048fcbe55f6849bb83a
Jul 08 03:24:13.956564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:13.956529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:13.956719 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:13.956673    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 03:24:13.956774 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:13.956736    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls podName:2e8f6056-9a9e-4532-8224-6189831b56af nodeName:}" failed. No retries permitted until 2026-07-08 03:24:14.956718117 +0000 UTC m=+270.998902763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-tw97b" (UID: "2e8f6056-9a9e-4532-8224-6189831b56af") : secret "samples-operator-tls" not found
Jul 08 03:24:14.057217 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.057180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:14.057361 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:14.057275    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:14.057361 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:14.057331    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls podName:78e7b8a1-4722-4273-b358-1dc16d59d957 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:15.057308006 +0000 UTC m=+271.099492653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-gvdbh" (UID: "78e7b8a1-4722-4273-b358-1dc16d59d957") : secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:14.244572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.244475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp" event={"ID":"68111e66-8ce8-4eec-b76b-363147c018eb","Type":"ContainerStarted","Data":"9d441837d7801d835640928ada2eddfe18f9bc5c5380f11c16817028564e4293"}
Jul 08 03:24:14.245638 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.245607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g" event={"ID":"88fab518-b439-4c75-85bf-52139e1bd0f5","Type":"ContainerStarted","Data":"fed67557e1f5791f2c3f6342d1817dc1f3fe788b3686832d1ffe4b19dff5949e"}
Jul 08 03:24:14.247005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.246982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4" event={"ID":"5c40feac-bd39-4fac-ac53-7c37fca4e846","Type":"ContainerStarted","Data":"3be8e4f9948fc8eb9f0c13eb56e0259e62b05124b83658a7c15e6514e41be24c"}
Jul 08 03:24:14.247092 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.247021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4" event={"ID":"5c40feac-bd39-4fac-ac53-7c37fca4e846","Type":"ContainerStarted","Data":"843b1eea7c5be0d1c25b1d992fcad9af3871a8db82bd4048fcbe55f6849bb83a"}
Jul 08 03:24:14.263753 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.263711    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-75bj4" podStartSLOduration=1.263698007 podStartE2EDuration="1.263698007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:14.262444882 +0000 UTC m=+270.304629555" watchObservedRunningTime="2026-07-08 03:24:14.263698007 +0000 UTC m=+270.305882667"
Jul 08 03:24:14.963698 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:14.963662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:14.964106 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:14.963835    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 03:24:14.964106 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:14.963913    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls podName:2e8f6056-9a9e-4532-8224-6189831b56af nodeName:}" failed. No retries permitted until 2026-07-08 03:24:16.96389154 +0000 UTC m=+273.006076212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-tw97b" (UID: "2e8f6056-9a9e-4532-8224-6189831b56af") : secret "samples-operator-tls" not found
Jul 08 03:24:15.064417 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:15.064378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:15.064596 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:15.064516    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:15.064596 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:15.064591    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls podName:78e7b8a1-4722-4273-b358-1dc16d59d957 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:17.064574275 +0000 UTC m=+273.106758944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-gvdbh" (UID: "78e7b8a1-4722-4273-b358-1dc16d59d957") : secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:15.269941 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:15.269889    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g" podStartSLOduration=0.838160469 podStartE2EDuration="2.26986989s" totalImagesPullingTime="1.431709421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:15.268031534 +0000 UTC m=+271.310216270" watchObservedRunningTime="2026-07-08 03:24:15.26986989 +0000 UTC m=+271.312054562"
Jul 08 03:24:15.367935 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:15.367901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:15.424639 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:15.424519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:16.260333 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:16.260237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp" event={"ID":"68111e66-8ce8-4eec-b76b-363147c018eb","Type":"ContainerStarted","Data":"261c86a3c844e6c6cdc4202239a65dd122205adb66cb4db2e8fdd4f8d9a575f4"}
Jul 08 03:24:16.261679 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:16.261656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mqq6g" event={"ID":"88fab518-b439-4c75-85bf-52139e1bd0f5","Type":"ContainerStarted","Data":"10a875643c774f6afe284eacf5fbb69ddf0d5fdc55e652181ca219d79a93c534"}
Jul 08 03:24:16.276444 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:16.276397    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp" podStartSLOduration=1.098819051 podStartE2EDuration="3.276384091s" totalImagesPullingTime="2.17756504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:16.275742396 +0000 UTC m=+272.317927070" watchObservedRunningTime="2026-07-08 03:24:16.276384091 +0000 UTC m=+272.318568789"
Jul 08 03:24:16.979276 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:16.979241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:16.979420 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:16.979347    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 03:24:16.979420 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:16.979402    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls podName:2e8f6056-9a9e-4532-8224-6189831b56af nodeName:}" failed. No retries permitted until 2026-07-08 03:24:20.979386018 +0000 UTC m=+277.021570664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-tw97b" (UID: "2e8f6056-9a9e-4532-8224-6189831b56af") : secret "samples-operator-tls" not found
Jul 08 03:24:17.079743 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:17.079710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:17.079896 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:17.079819    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:17.079896 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:17.079872    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls podName:78e7b8a1-4722-4273-b358-1dc16d59d957 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.079856741 +0000 UTC m=+277.122041388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-gvdbh" (UID: "78e7b8a1-4722-4273-b358-1dc16d59d957") : secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:17.119094 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:17.119054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:17.211724 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:17.211689    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:18.095462 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.095427    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"]
Jul 08 03:24:18.097491 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.097476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"
Jul 08 03:24:18.100121 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.100093    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 08 03:24:18.100847 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.100830    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5qqd5\""
Jul 08 03:24:18.100915 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.100830    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 08 03:24:18.111298 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.111275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"]
Jul 08 03:24:18.141894 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.141865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:18.186309 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.186267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42tp\" (UniqueName: \"kubernetes.io/projected/99ff6c39-1c63-4e9d-99df-3366f4979cdd-kube-api-access-f42tp\") pod \"migrator-d57d58cc9-cgmwc\" (UID: \"99ff6c39-1c63-4e9d-99df-3366f4979cdd\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"
Jul 08 03:24:18.194559 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.194532    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8p5ht"]
Jul 08 03:24:18.196904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.196887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.199268 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.199250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 03:24:18.199367 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.199269    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 03:24:18.199367 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.199290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kfwlq\""
Jul 08 03:24:18.199367 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.199323    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 03:24:18.199549 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.199536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 03:24:18.209266 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.209245    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8p5ht"]
Jul 08 03:24:18.238679 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.238650    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:18.287599 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.287578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f42tp\" (UniqueName: \"kubernetes.io/projected/99ff6c39-1c63-4e9d-99df-3366f4979cdd-kube-api-access-f42tp\") pod \"migrator-d57d58cc9-cgmwc\" (UID: \"99ff6c39-1c63-4e9d-99df-3366f4979cdd\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"
Jul 08 03:24:18.287705 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.287609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-crio-socket\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.287705 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.287630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.287705 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.287646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjccz\" (UniqueName: \"kubernetes.io/projected/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-api-access-pjccz\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.287820 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.287721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-data-volume\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.287820 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.287812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.295572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.295542    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42tp\" (UniqueName: \"kubernetes.io/projected/99ff6c39-1c63-4e9d-99df-3366f4979cdd-kube-api-access-f42tp\") pod \"migrator-d57d58cc9-cgmwc\" (UID: \"99ff6c39-1c63-4e9d-99df-3366f4979cdd\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"
Jul 08 03:24:18.303833 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.303809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:18.389025 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.388964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-data-volume\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.389117 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.389026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.389117 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.389077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-crio-socket\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.389117 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.389105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.389250 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.389128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjccz\" (UniqueName: \"kubernetes.io/projected/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-api-access-pjccz\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.389250 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.389174    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:18.389250 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.389203    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.389250 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.389237    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:18.889219633 +0000 UTC m=+274.931404281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:18.389382 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.389250    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-data-volume\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.389382 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.389261    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:18.889245512 +0000 UTC m=+274.931430162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.389382 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.389206    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-crio-socket\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.397409 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.397393    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjccz\" (UniqueName: \"kubernetes.io/projected/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-api-access-pjccz\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.405732 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.405712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"
Jul 08 03:24:18.532118 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.532088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc"]
Jul 08 03:24:18.535199 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:18.535174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ff6c39_1c63_4e9d_99df_3366f4979cdd.slice/crio-e55e739fda8b0589da6f396633211962a2bee3ebac172d81ec567596a8b536f3 WatchSource:0}: Error finding container e55e739fda8b0589da6f396633211962a2bee3ebac172d81ec567596a8b536f3: Status 404 returned error can't find the container with id e55e739fda8b0589da6f396633211962a2bee3ebac172d81ec567596a8b536f3
Jul 08 03:24:18.894219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.894162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.894501 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.894277    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.894501 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:18.894295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:18.894501 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.894334    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:19.894318376 +0000 UTC m=+275.936503023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.894501 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.894408    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:18.894666 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:18.894511    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:19.894493901 +0000 UTC m=+275.936678548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:19.182566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.182473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:19.239357 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.239313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:19.270849 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.270800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc" event={"ID":"99ff6c39-1c63-4e9d-99df-3366f4979cdd","Type":"ContainerStarted","Data":"e55e739fda8b0589da6f396633211962a2bee3ebac172d81ec567596a8b536f3"}
Jul 08 03:24:19.420537 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.420495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:19.481282 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.481199    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:19.886468 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.886443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:19.902844 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.902813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:19.902963 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.902921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:19.902963 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:19.902928    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:19.903083 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:19.902995    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.902980257 +0000 UTC m=+277.945164903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:19.903083 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:19.903024    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:19.903083 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:19.903067    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.903052402 +0000 UTC m=+277.945237054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:19.988838 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:19.988807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:20.279087 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:20.278996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc" event={"ID":"99ff6c39-1c63-4e9d-99df-3366f4979cdd","Type":"ContainerStarted","Data":"0b6f8f001fc8c900475530043a8173f8ba91ee9ebefb4bc5615a31fdee1d1986"}
Jul 08 03:24:20.279087 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:20.279070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc" event={"ID":"99ff6c39-1c63-4e9d-99df-3366f4979cdd","Type":"ContainerStarted","Data":"de597095322faaad9778203ef9fc207c94e6340621b0d2b15e2bca212a32f5f1"}
Jul 08 03:24:20.295224 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:20.295179    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cgmwc" podStartSLOduration=1.00788212 podStartE2EDuration="2.295166321s" totalImagesPullingTime="1.287284201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:20.294338423 +0000 UTC m=+276.336523091" watchObservedRunningTime="2026-07-08 03:24:20.295166321 +0000 UTC m=+276.337350989"
Jul 08 03:24:20.373667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:20.373628    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:20.986790 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:20.986764    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sz6hp_5796db27-474f-4e9e-9b0a-a3a219c600b4/dns-node-resolver/0.log"
Jul 08 03:24:21.013670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.013644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:21.013781 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.013769    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 03:24:21.013841 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.013830    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls podName:2e8f6056-9a9e-4532-8224-6189831b56af nodeName:}" failed. No retries permitted until 2026-07-08 03:24:29.013816679 +0000 UTC m=+285.056001326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-tw97b" (UID: "2e8f6056-9a9e-4532-8224-6189831b56af") : secret "samples-operator-tls" not found
Jul 08 03:24:21.114888 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.114852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:21.115063 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.114979    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:21.115063 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.115033    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls podName:78e7b8a1-4722-4273-b358-1dc16d59d957 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:29.115019703 +0000 UTC m=+285.157204351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-gvdbh" (UID: "78e7b8a1-4722-4273-b358-1dc16d59d957") : secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:21.523023 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.522988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:21.583638 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.583606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:21.921117 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.921067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:21.921305 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.921174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:21.921305 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.921191    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:21.921305 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.921260    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:25.921245362 +0000 UTC m=+281.963430013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:21.921305 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.921289    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:21.921441 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:21.921333    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:25.921320938 +0000 UTC m=+281.963505585 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:21.982425 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:21.982388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:22.186448 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:22.186375    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwssj_1f3445b3-8eb2-4137-8a58-f1637be7d7e4/node-ca/0.log"
Jul 08 03:24:22.406699 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:22.406666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:22.591796 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:22.591778    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4vvts_c1d4333f-55cc-414a-a948-48d70aef566e/serve-healthcheck-canary/0.log"
Jul 08 03:24:23.163541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.163512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:23.187328 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.187299    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-cgmwc_99ff6c39-1c63-4e9d-99df-3366f4979cdd/migrator/0.log"
Jul 08 03:24:23.378478 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.378450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:23.389877 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.389859    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-cgmwc_99ff6c39-1c63-4e9d-99df-3366f4979cdd/graceful-termination/0.log"
Jul 08 03:24:23.480589 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.480343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:23.543008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.542981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:23.588730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.588703    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-5lrfp_68111e66-8ce8-4eec-b76b-363147c018eb/kube-storage-version-migrator-operator/0.log"
Jul 08 03:24:23.770602 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.770521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:23.871049 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:23.871015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:24.693569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:24.693541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:25.259755 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:25.259713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:25.463081 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:25.463052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:25.662307 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:25.662271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:25.915532 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:25.915450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:25.956688 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:25.956662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:25.956786 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:25.956735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:25.956786 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:25.956780    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:25.956874 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:25.956824    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:25.956874 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:25.956836    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:33.956822313 +0000 UTC m=+289.999006959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:25.956874 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:25.956872    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls podName:600a1819-ff8e-4dc3-8f50-28e4b6e0b386 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:33.956860509 +0000 UTC m=+289.999045156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8p5ht" (UID: "600a1819-ff8e-4dc3-8f50-28e4b6e0b386") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:26.068931 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:26.068892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:26.866931 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:26.866894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:26.967503 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:26.967471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:26.989449 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:26.989417    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-r8rwb" podUID="a3458bec-0dc7-42ff-b199-84cbafc31675"
Jul 08 03:24:26.989558 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:26.989418    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z" podUID="5e7b6863-5540-4e0c-a79b-a725921fcd4e"
Jul 08 03:24:26.989558 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:26.989421    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" podUID="919205f6-7f4b-434c-9564-312f471eaf83"
Jul 08 03:24:27.067140 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.067110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:27.138079 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.137995    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:27.298703 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.298669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:24:27.298899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.298670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:24:27.298899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.298677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:24:27.460131 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.460034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:27.674356 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:27.674318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:28.066171 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:28.066131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:28.663588 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:28.663552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:29.057451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.057409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:29.083459 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.083431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:29.086056 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.086028    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e8f6056-9a9e-4532-8224-6189831b56af-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-tw97b\" (UID: \"2e8f6056-9a9e-4532-8224-6189831b56af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:29.183972 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.183916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:29.184131 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:29.184071    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:29.184170 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:29.184137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls podName:78e7b8a1-4722-4273-b358-1dc16d59d957 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:45.184119397 +0000 UTC m=+301.226304046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-gvdbh" (UID: "78e7b8a1-4722-4273-b358-1dc16d59d957") : secret "cluster-monitoring-operator-tls" not found
Jul 08 03:24:29.230706 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.230676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"
Jul 08 03:24:29.352734 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.352703    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b"]
Jul 08 03:24:29.457172 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.457146    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:29.785639 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.785561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:29.886336 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:29.886306    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:30.261452 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.261405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:30.294501 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.294471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:24:30.294673 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.294512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:24:30.294673 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:30.294640    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 03:24:30.294783 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:30.294723    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert podName:5e7b6863-5540-4e0c-a79b-a725921fcd4e nodeName:}" failed. No retries permitted until 2026-07-08 03:26:32.294701376 +0000 UTC m=+408.336886027 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-5h75z" (UID: "5e7b6863-5540-4e0c-a79b-a725921fcd4e") : secret "networking-console-plugin-cert" not found
Jul 08 03:24:30.294858 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.294841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:24:30.297397 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.297375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3458bec-0dc7-42ff-b199-84cbafc31675-metrics-tls\") pod \"dns-default-r8rwb\" (UID: \"a3458bec-0dc7-42ff-b199-84cbafc31675\") " pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:24:30.297659 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.297638    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"image-registry-66b449c4f7-sp8qb\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") " pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:24:30.301766 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.301740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gjrkj\""
Jul 08 03:24:30.301879 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.301858    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5t5kv\""
Jul 08 03:24:30.309541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.309525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:24:30.309689 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.309667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:24:30.309883 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.309856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b" event={"ID":"2e8f6056-9a9e-4532-8224-6189831b56af","Type":"ContainerStarted","Data":"77d2732c03a3b36156e3ea4a288cbefdc0affb5067cdca390462972370b67c0d"}
Jul 08 03:24:30.348921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.348856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:30.456689 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.456632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:30.471432 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.471394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b449c4f7-sp8qb"]
Jul 08 03:24:30.476043 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:30.476015    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919205f6_7f4b_434c_9564_312f471eaf83.slice/crio-e9f74f309125cf13981acbc8c2e6d4d1262b03a8a4614de3b2b2afce3e25548d WatchSource:0}: Error finding container e9f74f309125cf13981acbc8c2e6d4d1262b03a8a4614de3b2b2afce3e25548d: Status 404 returned error can't find the container with id e9f74f309125cf13981acbc8c2e6d4d1262b03a8a4614de3b2b2afce3e25548d
Jul 08 03:24:30.485752 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:30.485727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8rwb"]
Jul 08 03:24:30.488865 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:30.488836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3458bec_0dc7_42ff_b199_84cbafc31675.slice/crio-29045061abb81de24bacbb0d61947fe47284eedc38f8283a3ce97eb908a3d2aa WatchSource:0}: Error finding container 29045061abb81de24bacbb0d61947fe47284eedc38f8283a3ce97eb908a3d2aa: Status 404 returned error can't find the container with id 29045061abb81de24bacbb0d61947fe47284eedc38f8283a3ce97eb908a3d2aa
Jul 08 03:24:31.053678 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.053634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:31.314969 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.314872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8rwb" event={"ID":"a3458bec-0dc7-42ff-b199-84cbafc31675","Type":"ContainerStarted","Data":"29045061abb81de24bacbb0d61947fe47284eedc38f8283a3ce97eb908a3d2aa"}
Jul 08 03:24:31.316782 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.316750    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" event={"ID":"919205f6-7f4b-434c-9564-312f471eaf83","Type":"ContainerStarted","Data":"29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df"}
Jul 08 03:24:31.316921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.316798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" event={"ID":"919205f6-7f4b-434c-9564-312f471eaf83","Type":"ContainerStarted","Data":"e9f74f309125cf13981acbc8c2e6d4d1262b03a8a4614de3b2b2afce3e25548d"}
Jul 08 03:24:31.316921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.316892    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:24:31.337874 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.337818    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" podStartSLOduration=286.337798222 podStartE2EDuration="4m46.337798222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:31.335773093 +0000 UTC m=+287.377957764" watchObservedRunningTime="2026-07-08 03:24:31.337798222 +0000 UTC m=+287.379982893"
Jul 08 03:24:31.456252 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.456216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:31.861212 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:31.861177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:32.258509 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:32.258420    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:32.322452 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:32.322413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b" event={"ID":"2e8f6056-9a9e-4532-8224-6189831b56af","Type":"ContainerStarted","Data":"8059b566fd75561db33bcaec689b45d375f5ec48ca6a0580959138f18466c384"}
Jul 08 03:24:32.322882 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:32.322475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b" event={"ID":"2e8f6056-9a9e-4532-8224-6189831b56af","Type":"ContainerStarted","Data":"86295ec1ae3c734efc756829610a6b88f47ea36be448e66959e89db769bb17cc"}
Jul 08 03:24:32.339150 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:32.339096    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-tw97b" podStartSLOduration=17.303909523 podStartE2EDuration="19.339075648s" totalImagesPullingTime="2.035166125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:32.33832374 +0000 UTC m=+288.380508410" watchObservedRunningTime="2026-07-08 03:24:32.339075648 +0000 UTC m=+288.381260321"
Jul 08 03:24:32.655790 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:32.655760    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:33.064251 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.064223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:33.259311 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.259275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:33.327625 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.327541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8rwb" event={"ID":"a3458bec-0dc7-42ff-b199-84cbafc31675","Type":"ContainerStarted","Data":"e3e9b9c45a44050120f2717eaa026996250ad241135bc2863e6ac8c857baff42"}
Jul 08 03:24:33.327625 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.327596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8rwb" event={"ID":"a3458bec-0dc7-42ff-b199-84cbafc31675","Type":"ContainerStarted","Data":"47b5f4d6d3c3c54b7e5bd914582af6eddc6e23711cdc75c7f5c9bfb2f6edd372"}
Jul 08 03:24:33.328035 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.327688    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:24:33.346029 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.345977    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r8rwb" podStartSLOduration=251.357586308 podStartE2EDuration="4m13.345929463s" totalImagesPullingTime="1.988343155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:33.345419263 +0000 UTC m=+289.387603933" watchObservedRunningTime="2026-07-08 03:24:33.345929463 +0000 UTC m=+289.388114133"
Jul 08 03:24:33.608495 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.608411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:33.710451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.710413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:33.867640 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:33.867530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:34.026448 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.026414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:34.026614 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.026480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:34.026984 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.026940    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:34.028899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.028881    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/600a1819-ff8e-4dc3-8f50-28e4b6e0b386-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8p5ht\" (UID: \"600a1819-ff8e-4dc3-8f50-28e4b6e0b386\") " pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:34.106145 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.106080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8p5ht"
Jul 08 03:24:34.257899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.257802    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8p5ht"]
Jul 08 03:24:34.260442 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:34.260412    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600a1819_ff8e_4dc3_8f50_28e4b6e0b386.slice/crio-b85f58b7fadc270d4e3db606e9ae90d197330841b8577e339c99d76d2cea8fab WatchSource:0}: Error finding container b85f58b7fadc270d4e3db606e9ae90d197330841b8577e339c99d76d2cea8fab: Status 404 returned error can't find the container with id b85f58b7fadc270d4e3db606e9ae90d197330841b8577e339c99d76d2cea8fab
Jul 08 03:24:34.333423 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:34.333385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8p5ht" event={"ID":"600a1819-ff8e-4dc3-8f50-28e4b6e0b386","Type":"ContainerStarted","Data":"b85f58b7fadc270d4e3db606e9ae90d197330841b8577e339c99d76d2cea8fab"}
Jul 08 03:24:38.351009 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:38.350976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8p5ht" event={"ID":"600a1819-ff8e-4dc3-8f50-28e4b6e0b386","Type":"ContainerStarted","Data":"5b3d0acc600a6d1c06c0bd916a5bbedb8fc9815431549db9c64ba6fef6ce2dcc"}
Jul 08 03:24:39.356939 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:39.356903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8p5ht" event={"ID":"600a1819-ff8e-4dc3-8f50-28e4b6e0b386","Type":"ContainerStarted","Data":"8131ce7a47bdf8bab9e736f25df5750ffcea5122573e8e984ebe4896601d1884"}
Jul 08 03:24:39.415304 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:39.415261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:39.452736 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:39.452698    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8cxvd" podUID="b289c98c-c888-4672-9b09-03e12ae66c52"
Jul 08 03:24:39.479763 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:39.479726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:39.540603 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:39.540567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:39.696029 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:39.695925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:40.093091 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:40.093031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:40.166042 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:40.166006    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:40.286614 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:40.286578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:40.362338 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:40.362253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:41.290898 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.290863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:41.365593 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.365509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8p5ht" event={"ID":"600a1819-ff8e-4dc3-8f50-28e4b6e0b386","Type":"ContainerStarted","Data":"b9017fbb6d522d15236d929e0484938dca269adfb8452678bc40ae59d1394aed"}
Jul 08 03:24:41.384547 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.384503    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8p5ht" podStartSLOduration=16.664029159000002 podStartE2EDuration="23.384489611s" totalImagesPullingTime="6.720460452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:24:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:41.383184548 +0000 UTC m=+297.425369216" watchObservedRunningTime="2026-07-08 03:24:41.384489611 +0000 UTC m=+297.426674280"
Jul 08 03:24:41.489869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.489835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:41.591533 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.591501    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:41.691171 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.691086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:41.745586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:41.745556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:43.336377 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:43.336352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r8rwb"
Jul 08 03:24:44.333332 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.333301    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:24:44.334482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.334463    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:24:44.336225 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.336200    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 03:24:44.499008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.498872    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-jd2qv"]
Jul 08 03:24:44.516463 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.503368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:24:44.516463 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.505830    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 03:24:44.516463 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.506310    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 03:24:44.516463 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.506388    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fhmmn\""
Jul 08 03:24:44.517488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.517467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-jd2qv"]
Jul 08 03:24:44.609595 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.609529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8h4h\" (UniqueName: \"kubernetes.io/projected/0dd8ee21-1e1a-44f4-b223-611fa1759e2d-kube-api-access-p8h4h\") pod \"downloads-685bfb5f64-jd2qv\" (UID: \"0dd8ee21-1e1a-44f4-b223-611fa1759e2d\") " pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:24:44.639615 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.639588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:44.710800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.710768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8h4h\" (UniqueName: \"kubernetes.io/projected/0dd8ee21-1e1a-44f4-b223-611fa1759e2d-kube-api-access-p8h4h\") pod \"downloads-685bfb5f64-jd2qv\" (UID: \"0dd8ee21-1e1a-44f4-b223-611fa1759e2d\") " pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:24:44.719084 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.719062    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8h4h\" (UniqueName: \"kubernetes.io/projected/0dd8ee21-1e1a-44f4-b223-611fa1759e2d-kube-api-access-p8h4h\") pod \"downloads-685bfb5f64-jd2qv\" (UID: \"0dd8ee21-1e1a-44f4-b223-611fa1759e2d\") " pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:24:44.734271 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.734247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:44.812283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.812260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:24:44.911076 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.911050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:44.934254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:44.934227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-jd2qv"]
Jul 08 03:24:44.937458 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:44.937435    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd8ee21_1e1a_44f4_b223_611fa1759e2d.slice/crio-a958105866598b1ba050b186d9e82aff1e951138f826c9317bea4e8953d9717e WatchSource:0}: Error finding container a958105866598b1ba050b186d9e82aff1e951138f826c9317bea4e8953d9717e: Status 404 returned error can't find the container with id a958105866598b1ba050b186d9e82aff1e951138f826c9317bea4e8953d9717e
Jul 08 03:24:45.011731 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.011697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:45.214704 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.214610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:45.217183 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.217163    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e7b8a1-4722-4273-b358-1dc16d59d957-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-gvdbh\" (UID: \"78e7b8a1-4722-4273-b358-1dc16d59d957\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:45.224281 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.224263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zmtd9\""
Jul 08 03:24:45.233169 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.233150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"
Jul 08 03:24:45.355449 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.355421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh"]
Jul 08 03:24:45.358133 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:45.358105    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e7b8a1_4722_4273_b358_1dc16d59d957.slice/crio-213e4c574c029bdf6941e63dac5568bcdd43eebded3d07b78ebb0c210fec4a2b WatchSource:0}: Error finding container 213e4c574c029bdf6941e63dac5568bcdd43eebded3d07b78ebb0c210fec4a2b: Status 404 returned error can't find the container with id 213e4c574c029bdf6941e63dac5568bcdd43eebded3d07b78ebb0c210fec4a2b
Jul 08 03:24:45.378787 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.378755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-jd2qv" event={"ID":"0dd8ee21-1e1a-44f4-b223-611fa1759e2d","Type":"ContainerStarted","Data":"a958105866598b1ba050b186d9e82aff1e951138f826c9317bea4e8953d9717e"}
Jul 08 03:24:45.379710 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:45.379688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh" event={"ID":"78e7b8a1-4722-4273-b358-1dc16d59d957","Type":"ContainerStarted","Data":"213e4c574c029bdf6941e63dac5568bcdd43eebded3d07b78ebb0c210fec4a2b"}
Jul 08 03:24:46.941913 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:46.941875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:47.912787 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:47.912748    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"]
Jul 08 03:24:47.916078 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:47.916058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:47.918467 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:47.918443    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 08 03:24:47.918595 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:47.918466    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7bvwh\""
Jul 08 03:24:47.924703 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:47.924681    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"]
Jul 08 03:24:47.937168 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:47.937136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9afe4d53-cf8d-4505-a4eb-42f7146b96bf-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5w9fb\" (UID: \"9afe4d53-cf8d-4505-a4eb-42f7146b96bf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:48.038186 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.038152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9afe4d53-cf8d-4505-a4eb-42f7146b96bf-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5w9fb\" (UID: \"9afe4d53-cf8d-4505-a4eb-42f7146b96bf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:48.041091 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.041062    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9afe4d53-cf8d-4505-a4eb-42f7146b96bf-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5w9fb\" (UID: \"9afe4d53-cf8d-4505-a4eb-42f7146b96bf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:48.143457 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.143415    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:48.229440 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.229361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:48.373676 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.373566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"]
Jul 08 03:24:48.376396 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:48.376367    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afe4d53_cf8d_4505_a4eb_42f7146b96bf.slice/crio-5ac243ea47f4111b43e98946bcdd390c1981a4e5d75f314be54147cdc0961190 WatchSource:0}: Error finding container 5ac243ea47f4111b43e98946bcdd390c1981a4e5d75f314be54147cdc0961190: Status 404 returned error can't find the container with id 5ac243ea47f4111b43e98946bcdd390c1981a4e5d75f314be54147cdc0961190
Jul 08 03:24:48.378768 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.378747    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:24:48.393335 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.393305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb" event={"ID":"9afe4d53-cf8d-4505-a4eb-42f7146b96bf","Type":"ContainerStarted","Data":"5ac243ea47f4111b43e98946bcdd390c1981a4e5d75f314be54147cdc0961190"}
Jul 08 03:24:48.395181 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.395155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh" event={"ID":"78e7b8a1-4722-4273-b358-1dc16d59d957","Type":"ContainerStarted","Data":"ee4ff4333f6aecc53ef7f39d38da4841988f45e640be35fa271f188ce0b4f77e"}
Jul 08 03:24:48.412389 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:48.412339    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-gvdbh" podStartSLOduration=33.372803253 podStartE2EDuration="35.412321086s" totalImagesPullingTime="2.039517833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:48.410866379 +0000 UTC m=+304.453051049" watchObservedRunningTime="2026-07-08 03:24:48.412321086 +0000 UTC m=+304.454505758"
Jul 08 03:24:50.315650 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.315610    2580 patch_prober.go:28] interesting pod/image-registry-66b449c4f7-sp8qb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 03:24:50.316080 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.315676    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" podUID="919205f6-7f4b-434c-9564-312f471eaf83" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 03:24:50.403359 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.403323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb" event={"ID":"9afe4d53-cf8d-4505-a4eb-42f7146b96bf","Type":"ContainerStarted","Data":"7a3aab5a5b147cdf99d19e8a933a44137b4102e619e9ac2b23d6754b0f2623a2"}
Jul 08 03:24:50.403558 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.403539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:50.408522 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.408501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb"
Jul 08 03:24:50.420269 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.420232    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5w9fb" podStartSLOduration=2.194554794 podStartE2EDuration="3.420218848s" totalImagesPullingTime="1.225664054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:50.419864811 +0000 UTC m=+306.462049481" watchObservedRunningTime="2026-07-08 03:24:50.420218848 +0000 UTC m=+306.462403516"
Jul 08 03:24:50.451969 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.451919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:24:50.928320 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.928285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:50.977841 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.977803    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"]
Jul 08 03:24:50.980296 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.980274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:50.983911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.983756    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 03:24:50.983911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.983756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fw8nl\""
Jul 08 03:24:50.983911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.983753    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 08 03:24:50.983911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.983794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 08 03:24:50.989643 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.989615    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"]
Jul 08 03:24:50.999890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:50.999866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:51.063240 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.063209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.063409 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.063246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42e7dee9-22b3-431a-a812-6b4bf095145f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.063409 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.063273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvfwd\" (UniqueName: \"kubernetes.io/projected/42e7dee9-22b3-431a-a812-6b4bf095145f-kube-api-access-wvfwd\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.063504 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.063395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.164903 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.164857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.164903 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.164903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42e7dee9-22b3-431a-a812-6b4bf095145f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.165156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.164930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvfwd\" (UniqueName: \"kubernetes.io/projected/42e7dee9-22b3-431a-a812-6b4bf095145f-kube-api-access-wvfwd\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.165156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.164980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.165156 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:51.165023    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 08 03:24:51.165156 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:51.165105    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls podName:42e7dee9-22b3-431a-a812-6b4bf095145f nodeName:}" failed. No retries permitted until 2026-07-08 03:24:51.665084029 +0000 UTC m=+307.707268694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-smqbf" (UID: "42e7dee9-22b3-431a-a812-6b4bf095145f") : secret "prometheus-operator-tls" not found
Jul 08 03:24:51.165651 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.165623    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42e7dee9-22b3-431a-a812-6b4bf095145f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.167716 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.167695    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.175465 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.175432    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvfwd\" (UniqueName: \"kubernetes.io/projected/42e7dee9-22b3-431a-a812-6b4bf095145f-kube-api-access-wvfwd\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.669649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.669612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:51.670130 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:51.669762    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 08 03:24:51.670130 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:51.669824    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls podName:42e7dee9-22b3-431a-a812-6b4bf095145f nodeName:}" failed. No retries permitted until 2026-07-08 03:24:52.669808328 +0000 UTC m=+308.711992976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-smqbf" (UID: "42e7dee9-22b3-431a-a812-6b4bf095145f") : secret "prometheus-operator-tls" not found
Jul 08 03:24:51.803529 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:51.803495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:52.327119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.327085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:24:52.604081 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.603995    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:52.677927 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.677897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:52.680315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.680293    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/42e7dee9-22b3-431a-a812-6b4bf095145f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-smqbf\" (UID: \"42e7dee9-22b3-431a-a812-6b4bf095145f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:52.716279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.716252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:52.771848 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.771816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:52.790863 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.790840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"
Jul 08 03:24:52.930869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:52.930826    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf"]
Jul 08 03:24:52.933300 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:52.933258    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e7dee9_22b3_431a_a812_6b4bf095145f.slice/crio-5a4dee4ad5ebe535d14a74105e4e8476ec04b3ea84d3fdf7911d08166bd3250d WatchSource:0}: Error finding container 5a4dee4ad5ebe535d14a74105e4e8476ec04b3ea84d3fdf7911d08166bd3250d: Status 404 returned error can't find the container with id 5a4dee4ad5ebe535d14a74105e4e8476ec04b3ea84d3fdf7911d08166bd3250d
Jul 08 03:24:53.417247 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:53.417212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf" event={"ID":"42e7dee9-22b3-431a-a812-6b4bf095145f","Type":"ContainerStarted","Data":"5a4dee4ad5ebe535d14a74105e4e8476ec04b3ea84d3fdf7911d08166bd3250d"}
Jul 08 03:24:53.524833 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:53.524790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:53.578893 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:53.578857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:54.315474 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.315439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b8fd5689f-7sxg6"]
Jul 08 03:24:54.317940 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.317918    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.320698 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.320675    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 03:24:54.321827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.321496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 03:24:54.321827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.321630    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 03:24:54.321827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.321496    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 03:24:54.321827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.321496    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 03:24:54.321827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.321495    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j2b7c\""
Jul 08 03:24:54.326544 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.326523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 03:24:54.330238 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.330204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:54.332001 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.331979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8fd5689f-7sxg6"]
Jul 08 03:24:54.393347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h8qt\" (UniqueName: \"kubernetes.io/projected/026332f7-0884-4ea1-a23c-bf500d768aec-kube-api-access-2h8qt\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.393489 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-oauth-serving-cert\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.393489 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-trusted-ca-bundle\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.393489 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-serving-cert\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.393670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-console-config\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.393670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-oauth-config\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.393670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.393654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-service-ca\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.494670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.494648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-oauth-config\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.494783 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.494681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-service-ca\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.494783 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.494719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h8qt\" (UniqueName: \"kubernetes.io/projected/026332f7-0884-4ea1-a23c-bf500d768aec-kube-api-access-2h8qt\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.494783 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.494744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-oauth-serving-cert\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.495630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.495361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-trusted-ca-bundle\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.495630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.495432    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-service-ca\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.495630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.495443    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-oauth-serving-cert\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.495630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.495457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-serving-cert\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.495630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.495629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-console-config\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.496228 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.496187    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-trusted-ca-bundle\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.496315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.496229    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-console-config\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.497818 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.497796    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-oauth-config\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.498655 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.498633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-serving-cert\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.504314 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.504255    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h8qt\" (UniqueName: \"kubernetes.io/projected/026332f7-0884-4ea1-a23c-bf500d768aec-kube-api-access-2h8qt\") pod \"console-6b8fd5689f-7sxg6\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") " pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.630568 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.630529    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:24:54.723372 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.723312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:54.795207 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.795173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8fd5689f-7sxg6"]
Jul 08 03:24:54.799098 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:24:54.799072    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026332f7_0884_4ea1_a23c_bf500d768aec.slice/crio-3cf03c75d957de8cc0727c21e9a2a315a5978b7f0a458e77c2eec1d4236e6d6a WatchSource:0}: Error finding container 3cf03c75d957de8cc0727c21e9a2a315a5978b7f0a458e77c2eec1d4236e6d6a: Status 404 returned error can't find the container with id 3cf03c75d957de8cc0727c21e9a2a315a5978b7f0a458e77c2eec1d4236e6d6a
Jul 08 03:24:54.823534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.823462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:54.882770 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:54.882734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:55.318373 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:55.318328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:55.377517 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:55.377449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:55.426395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:55.426358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf" event={"ID":"42e7dee9-22b3-431a-a812-6b4bf095145f","Type":"ContainerStarted","Data":"a1ccdbe55f496516c262d9c0450fa3366b4ecb2a59665920742e88a2a74212f6"}
Jul 08 03:24:55.426548 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:55.426416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf" event={"ID":"42e7dee9-22b3-431a-a812-6b4bf095145f","Type":"ContainerStarted","Data":"eacf1f378d4783e6b3650a45134e43dd01847f6895ad93a87537e1e103ea9b90"}
Jul 08 03:24:55.427724 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:55.427687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8fd5689f-7sxg6" event={"ID":"026332f7-0884-4ea1-a23c-bf500d768aec","Type":"ContainerStarted","Data":"3cf03c75d957de8cc0727c21e9a2a315a5978b7f0a458e77c2eec1d4236e6d6a"}
Jul 08 03:24:55.443941 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:55.443893    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-smqbf" podStartSLOduration=3.937438742 podStartE2EDuration="5.443879372s" totalImagesPullingTime="1.50644063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:55.442865497 +0000 UTC m=+311.485050168" watchObservedRunningTime="2026-07-08 03:24:55.443879372 +0000 UTC m=+311.486064041"
Jul 08 03:24:57.098082 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.098048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:57.272821 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.272785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:57.360016 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.359914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:57.366015 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.365977    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6sk6p"]
Jul 08 03:24:57.371122 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.371091    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.373586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.373549    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 03:24:57.373927 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.373900    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 03:24:57.373927 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.373915    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 03:24:57.374223 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.374205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-msnx5\""
Jul 08 03:24:57.419091 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-node-exporter-wtmp\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-sys\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-root\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8tkf\" (UniqueName: \"kubernetes.io/projected/93745864-df22-4d7c-8893-266b89638f94-kube-api-access-r8tkf\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93745864-df22-4d7c-8893-266b89638f94-metrics-client-ca\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/93745864-df22-4d7c-8893-266b89638f94-node-exporter-textfile\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-tls\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/93745864-df22-4d7c-8893-266b89638f94-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.419566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.419528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.481280 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.481232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:57.520360 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-sys\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520360 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-root\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8tkf\" (UniqueName: \"kubernetes.io/projected/93745864-df22-4d7c-8893-266b89638f94-kube-api-access-r8tkf\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520408    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-sys\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93745864-df22-4d7c-8893-266b89638f94-metrics-client-ca\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520457    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-root\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/93745864-df22-4d7c-8893-266b89638f94-node-exporter-textfile\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520804 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-tls\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520804 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/93745864-df22-4d7c-8893-266b89638f94-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520804 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.520804 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-node-exporter-wtmp\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.521040 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:57.520804    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 03:24:57.521040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520860    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/93745864-df22-4d7c-8893-266b89638f94-node-exporter-wtmp\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.521040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.520868    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/93745864-df22-4d7c-8893-266b89638f94-node-exporter-textfile\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.521040 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:24:57.520879    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-tls podName:93745864-df22-4d7c-8893-266b89638f94 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:58.020859139 +0000 UTC m=+314.063043792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-tls") pod "node-exporter-6sk6p" (UID: "93745864-df22-4d7c-8893-266b89638f94") : secret "node-exporter-tls" not found
Jul 08 03:24:57.521239 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.521117    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93745864-df22-4d7c-8893-266b89638f94-metrics-client-ca\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.521283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.521263    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/93745864-df22-4d7c-8893-266b89638f94-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.523246 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.523226    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.529086 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.529068    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8tkf\" (UniqueName: \"kubernetes.io/projected/93745864-df22-4d7c-8893-266b89638f94-kube-api-access-r8tkf\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:57.668116 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.668026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:57.764296 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.764256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:57.866156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:57.866116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:58.025007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.024896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-tls\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:58.027830 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.027796    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93745864-df22-4d7c-8893-266b89638f94-node-exporter-tls\") pod \"node-exporter-6sk6p\" (UID: \"93745864-df22-4d7c-8893-266b89638f94\") " pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:58.048127 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.048039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:58.282198 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.282128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sk6p"
Jul 08 03:24:58.382894 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.380187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:58.484911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.484877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:58.600718 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.600684    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:58.973807 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:58.973722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:24:59.037556 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:24:59.037521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:00.309658 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.309617    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:00.410792 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.410755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:00.570240 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.570165    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"]
Jul 08 03:25:00.574273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.574254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.579039 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.578908    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 08 03:25:00.579039 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.578921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 08 03:25:00.579273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.579050    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2f1mraom9t4ii\""
Jul 08 03:25:00.585535 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.585514    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 08 03:25:00.585694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.585524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 08 03:25:00.588311 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.588293    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pgqgg\""
Jul 08 03:25:00.588885 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.588866    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 08 03:25:00.602363 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.602340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"]
Jul 08 03:25:00.650129 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-grpc-tls\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650298 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650298 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650298 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650441 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-metrics-client-ca\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650441 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650530 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shqfd\" (UniqueName: \"kubernetes.io/projected/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-kube-api-access-shqfd\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.650530 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.650503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-tls\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.659402 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.659371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:00.751331 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-metrics-client-ca\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751331 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751553 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shqfd\" (UniqueName: \"kubernetes.io/projected/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-kube-api-access-shqfd\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751553 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-tls\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751656 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-grpc-tls\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751656 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751753 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.751753 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.751693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.752181 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.752129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-metrics-client-ca\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.755105 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.755075    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.755231 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.755078    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-grpc-tls\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.755627 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.755604    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.755998 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.755934    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.756339 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.756317    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.756448 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.756427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-secret-thanos-querier-tls\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.758630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.758580    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:00.760994 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.760974    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shqfd\" (UniqueName: \"kubernetes.io/projected/4d592b34-d08a-4281-8b9f-7aaef99cb9c7-kube-api-access-shqfd\") pod \"thanos-querier-65756b4f8f-5wnvr\" (UID: \"4d592b34-d08a-4281-8b9f-7aaef99cb9c7\") " pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:00.884879 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:00.884806    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:01.788743 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.788703    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6b999598d-cxtrh"]
Jul 08 03:25:01.793246 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.793220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.797596 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797572    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 03:25:01.798110 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b999598d-cxtrh"]
Jul 08 03:25:01.798295 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797828    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 08 03:25:01.798394 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lz4j6\""
Jul 08 03:25:01.798394 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797875    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-59361hleveiof\""
Jul 08 03:25:01.798497 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797893    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 08 03:25:01.798497 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.797894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 08 03:25:01.860890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.860849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-secret-metrics-server-tls\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.861085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.860898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-secret-metrics-server-client-certs\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.861085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.860925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4e24342f-4c1b-493f-8872-c6b8d082b693-audit-log\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.861175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.861107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp27h\" (UniqueName: \"kubernetes.io/projected/4e24342f-4c1b-493f-8872-c6b8d082b693-kube-api-access-pp27h\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.861175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.861152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4e24342f-4c1b-493f-8872-c6b8d082b693-metrics-server-audit-profiles\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.861273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.861185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-client-ca-bundle\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.861273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.861256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e24342f-4c1b-493f-8872-c6b8d082b693-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.961846 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.961814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e24342f-4c1b-493f-8872-c6b8d082b693-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.961860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-secret-metrics-server-tls\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.961892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-secret-metrics-server-client-certs\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.961914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4e24342f-4c1b-493f-8872-c6b8d082b693-audit-log\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.962019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp27h\" (UniqueName: \"kubernetes.io/projected/4e24342f-4c1b-493f-8872-c6b8d082b693-kube-api-access-pp27h\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.962048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4e24342f-4c1b-493f-8872-c6b8d082b693-metrics-server-audit-profiles\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962319 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.962080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-client-ca-bundle\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962430 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.962402    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4e24342f-4c1b-493f-8872-c6b8d082b693-audit-log\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.962711 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.962687    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e24342f-4c1b-493f-8872-c6b8d082b693-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.963266 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.963241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4e24342f-4c1b-493f-8872-c6b8d082b693-metrics-server-audit-profiles\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.965110 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.965063    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-secret-metrics-server-client-certs\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.965202 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.965139    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-secret-metrics-server-tls\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.965202 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.965148    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e24342f-4c1b-493f-8872-c6b8d082b693-client-ca-bundle\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:01.977655 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:01.977630    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp27h\" (UniqueName: \"kubernetes.io/projected/4e24342f-4c1b-493f-8872-c6b8d082b693-kube-api-access-pp27h\") pod \"metrics-server-6b999598d-cxtrh\" (UID: \"4e24342f-4c1b-493f-8872-c6b8d082b693\") " pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:02.106317 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.106275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:02.121881 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.121847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:02.141045 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.141014    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"]
Jul 08 03:25:02.144607 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.144588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.147084 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.147064    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xw8fv\""
Jul 08 03:25:02.147200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.147115    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 08 03:25:02.152166 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.152143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"]
Jul 08 03:25:02.186440 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.186403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:02.264945 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.264907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhgw\" (UniqueName: \"kubernetes.io/projected/a80a4880-e7fb-48c9-9cf4-a218d2010776-kube-api-access-5qhgw\") pod \"monitoring-plugin-d5dcff7cf-ndlsp\" (UID: \"a80a4880-e7fb-48c9-9cf4-a218d2010776\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.265138 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.265046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a80a4880-e7fb-48c9-9cf4-a218d2010776-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ndlsp\" (UID: \"a80a4880-e7fb-48c9-9cf4-a218d2010776\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.348802 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.348746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:02.365862 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.365775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a80a4880-e7fb-48c9-9cf4-a218d2010776-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ndlsp\" (UID: \"a80a4880-e7fb-48c9-9cf4-a218d2010776\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.366040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.365868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhgw\" (UniqueName: \"kubernetes.io/projected/a80a4880-e7fb-48c9-9cf4-a218d2010776-kube-api-access-5qhgw\") pod \"monitoring-plugin-d5dcff7cf-ndlsp\" (UID: \"a80a4880-e7fb-48c9-9cf4-a218d2010776\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.368843 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.368819    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a80a4880-e7fb-48c9-9cf4-a218d2010776-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ndlsp\" (UID: \"a80a4880-e7fb-48c9-9cf4-a218d2010776\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.374601 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.374556    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhgw\" (UniqueName: \"kubernetes.io/projected/a80a4880-e7fb-48c9-9cf4-a218d2010776-kube-api-access-5qhgw\") pod \"monitoring-plugin-d5dcff7cf-ndlsp\" (UID: \"a80a4880-e7fb-48c9-9cf4-a218d2010776\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.456052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.456025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:02.559539 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.559498    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:02.563239 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.563210    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b54847447-fr222"]
Jul 08 03:25:02.568652 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.568632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.571413 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.571382    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mcfs2\""
Jul 08 03:25:02.571564 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.571549    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 08 03:25:02.571685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.571640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 08 03:25:02.571862 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.571846    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 08 03:25:02.571938 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.571876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 08 03:25:02.572594 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.572536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 08 03:25:02.577113 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.576943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 08 03:25:02.583390 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.583359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b54847447-fr222"]
Jul 08 03:25:02.624482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.624401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:02.667867 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.667825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-telemeter-client-tls\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.667867 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.667862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.668085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.667941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84d4m\" (UniqueName: \"kubernetes.io/projected/afeb9812-9020-45b3-b778-66386d98814f-kube-api-access-84d4m\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.668085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.668011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-secret-telemeter-client\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.668085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.668051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.668219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.668148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-serving-certs-ca-bundle\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.668219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.668177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-metrics-client-ca\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.668304 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.668225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-federate-client-tls\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.768725 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-serving-certs-ca-bundle\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.768908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-metrics-client-ca\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.768908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-federate-client-tls\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.768908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-telemeter-client-tls\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.768908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.768908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84d4m\" (UniqueName: \"kubernetes.io/projected/afeb9812-9020-45b3-b778-66386d98814f-kube-api-access-84d4m\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.769200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-secret-telemeter-client\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.769200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.768993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.769601 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.769544    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-serving-certs-ca-bundle\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.769818 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.769704    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-metrics-client-ca\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.770066 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.770029    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afeb9812-9020-45b3-b778-66386d98814f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.771776 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.771745    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-federate-client-tls\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.771876 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.771814    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-telemeter-client-tls\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.772172 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.772150    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-secret-telemeter-client\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.772261 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.772214    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afeb9812-9020-45b3-b778-66386d98814f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.779343 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.779323    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84d4m\" (UniqueName: \"kubernetes.io/projected/afeb9812-9020-45b3-b778-66386d98814f-kube-api-access-84d4m\") pod \"telemeter-client-b54847447-fr222\" (UID: \"afeb9812-9020-45b3-b778-66386d98814f\") " pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:02.881281 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:02.881210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b54847447-fr222"
Jul 08 03:25:03.591845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.591807    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 03:25:03.598437 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.598349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:03.602922 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.602900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.605543 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.605517    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 03:25:03.605543 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.605531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 03:25:03.605867 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.605847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6ro4mm6jnr8gd\""
Jul 08 03:25:03.606035 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.605890    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 03:25:03.606035 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.605849    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 03:25:03.606175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.606088    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 03:25:03.606319 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.606287    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ws7rl\""
Jul 08 03:25:03.607013 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.606852    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 03:25:03.607292 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.607139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 03:25:03.607292 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.607149    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 03:25:03.607689 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.607671    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 03:25:03.608073 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.607877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 03:25:03.608073 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.607944    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 03:25:03.608206 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.608156    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 03:25:03.608673 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.608507    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 03:25:03.611282 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.611053    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 03:25:03.618976 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.617899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 03:25:03.676155 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676327 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676327 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/34aa8295-276e-4026-90e9-aaf5d8277e33-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676327 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676327 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-config\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/34aa8295-276e-4026-90e9-aaf5d8277e33-config-out\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccj8z\" (UniqueName: \"kubernetes.io/projected/34aa8295-276e-4026-90e9-aaf5d8277e33-kube-api-access-ccj8z\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.676746 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-web-config\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.677426 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.677426 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.677426 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.677426 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.677426 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.676921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777590 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/34aa8295-276e-4026-90e9-aaf5d8277e33-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777590 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-config\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/34aa8295-276e-4026-90e9-aaf5d8277e33-config-out\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.777827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccj8z\" (UniqueName: \"kubernetes.io/projected/34aa8295-276e-4026-90e9-aaf5d8277e33-kube-api-access-ccj8z\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-web-config\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.777980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.778254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.780845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778722    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.780845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.778744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.780845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.779924    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.780845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.780601    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.780845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.780652    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.781691 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.781453    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.782592 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.782078    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/34aa8295-276e-4026-90e9-aaf5d8277e33-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.782592 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.782556    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.783328 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.783298    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.783681 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.783657    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.784029 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.784007    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-config\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.784274 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.784250    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-web-config\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.784988 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.784930    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/34aa8295-276e-4026-90e9-aaf5d8277e33-config-out\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.785276 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.785256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.786271 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.786224    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.787736 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.787349    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.788154 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.788130    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/34aa8295-276e-4026-90e9-aaf5d8277e33-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.788634 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.788590    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.790488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.790450    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/34aa8295-276e-4026-90e9-aaf5d8277e33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.790581 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.790529    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccj8z\" (UniqueName: \"kubernetes.io/projected/34aa8295-276e-4026-90e9-aaf5d8277e33-kube-api-access-ccj8z\") pod \"prometheus-k8s-0\" (UID: \"34aa8295-276e-4026-90e9-aaf5d8277e33\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.924007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.923894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:03.960129 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:03.960025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:04.033101 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:04.033061    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:04.343316 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:04.343280    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8fd5689f-7sxg6"]
Jul 08 03:25:04.345740 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:04.345714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:04.856879 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:25:04.856020    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93745864_df22_4d7c_8893_266b89638f94.slice/crio-32b8f60a4b73e155d26cba1a806ccc7f9d0dfac5062e24190c441899392efdd5 WatchSource:0}: Error finding container 32b8f60a4b73e155d26cba1a806ccc7f9d0dfac5062e24190c441899392efdd5: Status 404 returned error can't find the container with id 32b8f60a4b73e155d26cba1a806ccc7f9d0dfac5062e24190c441899392efdd5
Jul 08 03:25:05.061105 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.057260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b999598d-cxtrh"]
Jul 08 03:25:05.063481 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:25:05.063388    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e24342f_4c1b_493f_8872_c6b8d082b693.slice/crio-6ff30825a588198f79a7640160e4e4377861c60ab7f836484eaaf9a9743d3283 WatchSource:0}: Error finding container 6ff30825a588198f79a7640160e4e4377861c60ab7f836484eaaf9a9743d3283: Status 404 returned error can't find the container with id 6ff30825a588198f79a7640160e4e4377861c60ab7f836484eaaf9a9743d3283
Jul 08 03:25:05.072186 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.071891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"]
Jul 08 03:25:05.075151 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:25:05.075115    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80a4880_e7fb_48c9_9cf4_a218d2010776.slice/crio-1456334529cd927def3e2f308026750afb4f499c27fa02808dba3b7214139644 WatchSource:0}: Error finding container 1456334529cd927def3e2f308026750afb4f499c27fa02808dba3b7214139644: Status 404 returned error can't find the container with id 1456334529cd927def3e2f308026750afb4f499c27fa02808dba3b7214139644
Jul 08 03:25:05.319463 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.319435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b54847447-fr222"]
Jul 08 03:25:05.323644 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.323491    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"]
Jul 08 03:25:05.323644 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.323554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 03:25:05.326650 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:25:05.326622    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafeb9812_9020_45b3_b778_66386d98814f.slice/crio-0e62f272074cd58d1781793acec8d5699dde04ef5a03f2911bd649e99c9e3e4f WatchSource:0}: Error finding container 0e62f272074cd58d1781793acec8d5699dde04ef5a03f2911bd649e99c9e3e4f: Status 404 returned error can't find the container with id 0e62f272074cd58d1781793acec8d5699dde04ef5a03f2911bd649e99c9e3e4f
Jul 08 03:25:05.331185 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:25:05.330640    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34aa8295_276e_4026_90e9_aaf5d8277e33.slice/crio-ad9051099e9394a376a7912a946341247440a2b2c79f16c92af72d906df2801d WatchSource:0}: Error finding container ad9051099e9394a376a7912a946341247440a2b2c79f16c92af72d906df2801d: Status 404 returned error can't find the container with id ad9051099e9394a376a7912a946341247440a2b2c79f16c92af72d906df2801d
Jul 08 03:25:05.470933 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.470877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8fd5689f-7sxg6" event={"ID":"026332f7-0884-4ea1-a23c-bf500d768aec","Type":"ContainerStarted","Data":"7095c805233877d4ec6219682f4be19ecddd5bef947f6c214c34efc1c62540ee"}
Jul 08 03:25:05.476155 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.476056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b54847447-fr222" event={"ID":"afeb9812-9020-45b3-b778-66386d98814f","Type":"ContainerStarted","Data":"0e62f272074cd58d1781793acec8d5699dde04ef5a03f2911bd649e99c9e3e4f"}
Jul 08 03:25:05.487035 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.486882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"33d649816e7b7f49930ac5ee816c7156789df0cfa85951f53c8b95af04d1803f"}
Jul 08 03:25:05.488884 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.488836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b8fd5689f-7sxg6" podStartSLOduration=1.434436118 podStartE2EDuration="11.48882017s" totalImagesPullingTime="10.054384052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:05.488096233 +0000 UTC m=+321.530280903" watchObservedRunningTime="2026-07-08 03:25:05.48882017 +0000 UTC m=+321.531004839"
Jul 08 03:25:05.489057 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.488932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sk6p" event={"ID":"93745864-df22-4d7c-8893-266b89638f94","Type":"ContainerStarted","Data":"32b8f60a4b73e155d26cba1a806ccc7f9d0dfac5062e24190c441899392efdd5"}
Jul 08 03:25:05.490618 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.490596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh" event={"ID":"4e24342f-4c1b-493f-8872-c6b8d082b693","Type":"ContainerStarted","Data":"6ff30825a588198f79a7640160e4e4377861c60ab7f836484eaaf9a9743d3283"}
Jul 08 03:25:05.492446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.492422    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-jd2qv" event={"ID":"0dd8ee21-1e1a-44f4-b223-611fa1759e2d","Type":"ContainerStarted","Data":"3b989150f15cc33423176e1da636b961ca3e274bdfae509b2ce92d931b698854"}
Jul 08 03:25:05.492795 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.492780    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:25:05.494492 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.494468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"ad9051099e9394a376a7912a946341247440a2b2c79f16c92af72d906df2801d"}
Jul 08 03:25:05.496133 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.496106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp" event={"ID":"a80a4880-e7fb-48c9-9cf4-a218d2010776","Type":"ContainerStarted","Data":"1456334529cd927def3e2f308026750afb4f499c27fa02808dba3b7214139644"}
Jul 08 03:25:05.501139 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.501122    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-jd2qv"
Jul 08 03:25:05.510017 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:05.509191    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-jd2qv" podStartSLOduration=1.556527064 podStartE2EDuration="21.509177598s" totalImagesPullingTime="19.952650534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:05.508207654 +0000 UTC m=+321.550392326" watchObservedRunningTime="2026-07-08 03:25:05.509177598 +0000 UTC m=+321.551362268"
Jul 08 03:25:06.512034 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:06.510684    2580 generic.go:356] "Generic (PLEG): container finished" podID="93745864-df22-4d7c-8893-266b89638f94" containerID="90e5552b0d530b3002703b3e870edc73c80a5d5cfffecf18aa97bf4226d2cf29" exitCode=0
Jul 08 03:25:06.512034 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:06.511781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sk6p" event={"ID":"93745864-df22-4d7c-8893-266b89638f94","Type":"ContainerDied","Data":"90e5552b0d530b3002703b3e870edc73c80a5d5cfffecf18aa97bf4226d2cf29"}
Jul 08 03:25:06.969066 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:06.968239    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66b449c4f7-sp8qb"]
Jul 08 03:25:07.054352 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:07.054312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:07.201404 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:07.201367    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:07.261246 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:07.261207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:07.523134 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:07.523000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sk6p" event={"ID":"93745864-df22-4d7c-8893-266b89638f94","Type":"ContainerStarted","Data":"53651a2ea081cebe3e26ed2699c8002a52c62bcb7df76da52ff8fd934b6bc2fa"}
Jul 08 03:25:07.523134 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:07.523054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sk6p" event={"ID":"93745864-df22-4d7c-8893-266b89638f94","Type":"ContainerStarted","Data":"13ef433fc282738da8a2751d134268b9bd56fb6939061a92e6e960166c5989fe"}
Jul 08 03:25:07.543326 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:07.543269    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6sk6p" podStartSLOduration=9.502114864 podStartE2EDuration="10.543247846s" totalImagesPullingTime="954.73438ms" totalInitContainerRuntime="86.398602ms" isStatefulPod=true podCreationTimestamp="2026-07-08 03:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:07.541494509 +0000 UTC m=+323.583679181" watchObservedRunningTime="2026-07-08 03:25:07.543247846 +0000 UTC m=+323.585432515"
Jul 08 03:25:08.127901 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:08.127794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:08.182349 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:08.182197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:11.549884 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.548856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"a80ef05f6ac9444901a6bab1252e65fed2872552df9e1d2e8f83e1efbf9312ae"}
Jul 08 03:25:11.552136 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.552106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp" event={"ID":"a80a4880-e7fb-48c9-9cf4-a218d2010776","Type":"ContainerStarted","Data":"64799b027da2d41eb36a1f4ae6ce5eb2a1b8bef88a86d8b1f1b1f5bf18035e56"}
Jul 08 03:25:11.553151 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.552866    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:11.554425 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.554172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b54847447-fr222" event={"ID":"afeb9812-9020-45b3-b778-66386d98814f","Type":"ContainerStarted","Data":"95546cacfd7d6959d72d7f67807f62b289bd45c70032ec2fe539dacf510c106c"}
Jul 08 03:25:11.557817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.556836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"e20b85ba5e9e5e557c4c1a2775bb24472f96c6ffe37202036a8747ba016b6db2"}
Jul 08 03:25:11.559787 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.559299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh" event={"ID":"4e24342f-4c1b-493f-8872-c6b8d082b693","Type":"ContainerStarted","Data":"371a66ea4440cd22144d777507648b24717bd1e44fcc7c3c5cd4ab4abb67319d"}
Jul 08 03:25:11.560647 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.560619    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp"
Jul 08 03:25:11.604317 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.604270    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh" podStartSLOduration=4.361590063 podStartE2EDuration="10.604249858s" totalImagesPullingTime="6.242659795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:11.602392824 +0000 UTC m=+327.644577506" watchObservedRunningTime="2026-07-08 03:25:11.604249858 +0000 UTC m=+327.646434528"
Jul 08 03:25:11.614933 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.614874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:11.620243 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.620180    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ndlsp" podStartSLOduration=3.3818742840000002 podStartE2EDuration="9.620161392s" totalImagesPullingTime="6.238287108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:11.61893983 +0000 UTC m=+327.661124513" watchObservedRunningTime="2026-07-08 03:25:11.620161392 +0000 UTC m=+327.662346040"
Jul 08 03:25:11.683668 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:11.683619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:12.566388 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.566308    2580 generic.go:356] "Generic (PLEG): container finished" podID="34aa8295-276e-4026-90e9-aaf5d8277e33" containerID="a80ef05f6ac9444901a6bab1252e65fed2872552df9e1d2e8f83e1efbf9312ae" exitCode=0
Jul 08 03:25:12.566904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.566389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerDied","Data":"a80ef05f6ac9444901a6bab1252e65fed2872552df9e1d2e8f83e1efbf9312ae"}
Jul 08 03:25:12.572320 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.572009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b54847447-fr222" event={"ID":"afeb9812-9020-45b3-b778-66386d98814f","Type":"ContainerStarted","Data":"e921e5bf90f85b770a06dc65d1fea0825f5221b7acb18cb8422999f28501fd5b"}
Jul 08 03:25:12.572320 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.572057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b54847447-fr222" event={"ID":"afeb9812-9020-45b3-b778-66386d98814f","Type":"ContainerStarted","Data":"eaa0f66447b6da0ea24d9aaa7b045a76b273003c9e4c7122224b4a7b1d018460"}
Jul 08 03:25:12.576683 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.576570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"9b3eb4b6ed3c1f49e17f3354914da66130c43ed1cbaa3675ef8b71d7e0dab35b"}
Jul 08 03:25:12.576683 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.576642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"fd52a093260b1b2616d11bd923c2852c2b4356c2d7fba91cf2186edf195c4f7a"}
Jul 08 03:25:12.615903 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:12.614428    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b54847447-fr222" podStartSLOduration=4.629713115 podStartE2EDuration="10.614409579s" totalImagesPullingTime="5.984696464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:12.613099269 +0000 UTC m=+328.655283939" watchObservedRunningTime="2026-07-08 03:25:12.614409579 +0000 UTC m=+328.656594249"
Jul 08 03:25:13.481754 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.481722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:13.582299 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.582251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:13.589567 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.589484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"3ea117fabf7234ab1e39eef47054adafec7183c1e9c2523c856acb8cbdeab1f8"}
Jul 08 03:25:13.589567 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.589569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"8bb1f0ae991cb12faf9a44ebcb3da8b3780525be1ea569583eb7b1978c63b7de"}
Jul 08 03:25:13.589844 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.589613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" event={"ID":"4d592b34-d08a-4281-8b9f-7aaef99cb9c7","Type":"ContainerStarted","Data":"9831a04bc86f3093f4bf41b28b059dd571bf8a7ebfcdc2873e82c1c7ba235f1f"}
Jul 08 03:25:13.589844 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.589825    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:13.613254 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:13.613205    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr" podStartSLOduration=6.304452114 podStartE2EDuration="13.613188929s" totalImagesPullingTime="7.308736815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:25:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:13.611247809 +0000 UTC m=+329.653432483" watchObservedRunningTime="2026-07-08 03:25:13.613188929 +0000 UTC m=+329.655373597"
Jul 08 03:25:14.137098 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:14.137069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:14.189735 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:14.189701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:14.631424 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:14.631395    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:25:16.084817 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:16.084772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:16.184460 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:16.184408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:17.175302 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.175281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:17.277157 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.277118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:17.377473 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.377438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:17.612745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.612654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"58cdbaedfcbb3b5c41fca299fa607ff7c53d4a1dee2f9dddb7a2a88eac90d5ac"}
Jul 08 03:25:17.612745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.612738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"fd4f6182e334575ad4cbcdf508dafdd73a990d465e803d82eb54f191a450e670"}
Jul 08 03:25:17.613040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.612777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"d232054a10ecf836dd814a0fc018a45ecf3a8b026e21ffb1432f4c6cb3c221b6"}
Jul 08 03:25:17.613040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.612813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"a966bfa81f62c93a8b2057fb7933de8a53e8919c0fd6fa06d5ebb3bcebe8a449"}
Jul 08 03:25:17.613040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.612848    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"a449694fbc38980e248224959c11aca7ea73ef8b5712b8fcc250b5f8440d8f27"}
Jul 08 03:25:17.613040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.612881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"34aa8295-276e-4026-90e9-aaf5d8277e33","Type":"ContainerStarted","Data":"c717274235d2c392f4c3a047c0ace3aadc8e34889f71ee0cec2842c6b69e34a8"}
Jul 08 03:25:17.650374 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:17.650314    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.38460426 podStartE2EDuration="14.65028985s" totalImagesPullingTime="10.233768154s" totalInitContainerRuntime="31.917436ms" isStatefulPod=false podCreationTimestamp="2026-07-08 03:25:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:17.647194806 +0000 UTC m=+333.689379474" watchObservedRunningTime="2026-07-08 03:25:17.65028985 +0000 UTC m=+333.692474522"
Jul 08 03:25:18.924439 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:18.924410    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:25:19.600997 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:19.600945    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65756b4f8f-5wnvr"
Jul 08 03:25:22.106781 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:22.106742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:22.107215 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:22.106802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:24.252993 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:24.252942    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4vvts_c1d4333f-55cc-414a-a948-48d70aef566e/serve-healthcheck-canary/0.log"
Jul 08 03:25:24.740461 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:24.740417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:24.793217 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:24.793155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:24.856400 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:24.856368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:26.600042 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:26.600005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:26.659828 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:26.659788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:30.511073 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.511010    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b8fd5689f-7sxg6" podUID="026332f7-0884-4ea1-a23c-bf500d768aec" containerName="console" containerID="cri-o://7095c805233877d4ec6219682f4be19ecddd5bef947f6c214c34efc1c62540ee" gracePeriod=15
Jul 08 03:25:30.671834 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.671802    2580 generic.go:356] "Generic (PLEG): container finished" podID="026332f7-0884-4ea1-a23c-bf500d768aec" containerID="7095c805233877d4ec6219682f4be19ecddd5bef947f6c214c34efc1c62540ee" exitCode=0
Jul 08 03:25:30.672015 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.671867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8fd5689f-7sxg6" event={"ID":"026332f7-0884-4ea1-a23c-bf500d768aec","Type":"ContainerDied","Data":"7095c805233877d4ec6219682f4be19ecddd5bef947f6c214c34efc1c62540ee"}
Jul 08 03:25:30.783346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.783282    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:25:30.853275 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-trusted-ca-bundle\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853275 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853287    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-oauth-config\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853311    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h8qt\" (UniqueName: \"kubernetes.io/projected/026332f7-0884-4ea1-a23c-bf500d768aec-kube-api-access-2h8qt\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853347    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-service-ca\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-oauth-serving-cert\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-serving-cert\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-console-config\") pod \"026332f7-0884-4ea1-a23c-bf500d768aec\" (UID: \"026332f7-0884-4ea1-a23c-bf500d768aec\") "
Jul 08 03:25:30.853848 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853814    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:30.854025 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.853995    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-service-ca" (OuterVolumeSpecName: "service-ca") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:30.854129 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.854108    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:30.854263 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.854126    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-console-config" (OuterVolumeSpecName: "console-config") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:30.855983 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.855938    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:30.855983 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.855944    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026332f7-0884-4ea1-a23c-bf500d768aec-kube-api-access-2h8qt" (OuterVolumeSpecName: "kube-api-access-2h8qt") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "kube-api-access-2h8qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:30.856121 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.856067    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "026332f7-0884-4ea1-a23c-bf500d768aec" (UID: "026332f7-0884-4ea1-a23c-bf500d768aec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:30.955207 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955180    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-trusted-ca-bundle\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:30.955207 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955204    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-oauth-config\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:30.955207 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955213    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h8qt\" (UniqueName: \"kubernetes.io/projected/026332f7-0884-4ea1-a23c-bf500d768aec-kube-api-access-2h8qt\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:30.955207 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955223    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-service-ca\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:30.955458 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955231    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-oauth-serving-cert\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:30.955458 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955239    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/026332f7-0884-4ea1-a23c-bf500d768aec-console-serving-cert\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:30.955458 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:30.955248    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/026332f7-0884-4ea1-a23c-bf500d768aec-console-config\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:31.677122 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:31.677093    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8fd5689f-7sxg6"
Jul 08 03:25:31.677122 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:31.677114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8fd5689f-7sxg6" event={"ID":"026332f7-0884-4ea1-a23c-bf500d768aec","Type":"ContainerDied","Data":"3cf03c75d957de8cc0727c21e9a2a315a5978b7f0a458e77c2eec1d4236e6d6a"}
Jul 08 03:25:31.677586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:31.677175    2580 scope.go:117] "RemoveContainer" containerID="7095c805233877d4ec6219682f4be19ecddd5bef947f6c214c34efc1c62540ee"
Jul 08 03:25:31.697100 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:31.697079    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8fd5689f-7sxg6"]
Jul 08 03:25:31.701234 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:31.701216    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b8fd5689f-7sxg6"]
Jul 08 03:25:32.011208 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.011110    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" podUID="919205f6-7f4b-434c-9564-312f471eaf83" containerName="registry" containerID="cri-o://29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df" gracePeriod=30
Jul 08 03:25:32.308055 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.308028    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:25:32.373188 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373154    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-registry-certificates\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373198    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-installation-pull-secrets\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373243    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-trusted-ca\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373288    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/919205f6-7f4b-434c-9564-312f471eaf83-ca-trust-extracted\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373305    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f77sm\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-kube-api-access-f77sm\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-bound-sa-token\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373373    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-image-registry-private-configuration\") pod \"919205f6-7f4b-434c-9564-312f471eaf83\" (UID: \"919205f6-7f4b-434c-9564-312f471eaf83\") "
Jul 08 03:25:32.373720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373582    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:32.373720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373652    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-registry-certificates\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.373815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.373732    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:32.376333 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.376302    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-kube-api-access-f77sm" (OuterVolumeSpecName: "kube-api-access-f77sm") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "kube-api-access-f77sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:32.376333 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.376315    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:32.376531 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.376381    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:32.376531 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.376388    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:32.376636 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.376528    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:32.383221 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.383198    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919205f6-7f4b-434c-9564-312f471eaf83-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "919205f6-7f4b-434c-9564-312f471eaf83" (UID: "919205f6-7f4b-434c-9564-312f471eaf83"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:25:32.457904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.457870    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026332f7-0884-4ea1-a23c-bf500d768aec" path="/var/lib/kubelet/pods/026332f7-0884-4ea1-a23c-bf500d768aec/volumes"
Jul 08 03:25:32.474154 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474128    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-installation-pull-secrets\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.474154 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474149    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-registry-tls\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.474279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474159    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/919205f6-7f4b-434c-9564-312f471eaf83-trusted-ca\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.474279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474167    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/919205f6-7f4b-434c-9564-312f471eaf83-ca-trust-extracted\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.474279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474175    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f77sm\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-kube-api-access-f77sm\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.474279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474183    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/919205f6-7f4b-434c-9564-312f471eaf83-bound-sa-token\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.474279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.474192    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/919205f6-7f4b-434c-9564-312f471eaf83-image-registry-private-configuration\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:25:32.682151 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.682119    2580 generic.go:356] "Generic (PLEG): container finished" podID="919205f6-7f4b-434c-9564-312f471eaf83" containerID="29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df" exitCode=0
Jul 08 03:25:32.682624 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.682163    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb"
Jul 08 03:25:32.682624 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.682219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" event={"ID":"919205f6-7f4b-434c-9564-312f471eaf83","Type":"ContainerDied","Data":"29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df"}
Jul 08 03:25:32.682624 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.682268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b449c4f7-sp8qb" event={"ID":"919205f6-7f4b-434c-9564-312f471eaf83","Type":"ContainerDied","Data":"e9f74f309125cf13981acbc8c2e6d4d1262b03a8a4614de3b2b2afce3e25548d"}
Jul 08 03:25:32.682624 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.682299    2580 scope.go:117] "RemoveContainer" containerID="29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df"
Jul 08 03:25:32.690439 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.690419    2580 scope.go:117] "RemoveContainer" containerID="29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df"
Jul 08 03:25:32.690700 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:25:32.690676    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df\": container with ID starting with 29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df not found: ID does not exist" containerID="29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df"
Jul 08 03:25:32.690768 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.690708    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df"} err="failed to get container status \"29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df\": rpc error: code = NotFound desc = could not find container \"29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df\": container with ID starting with 29b95ee7423d2c34c8230227d44040f53ae7b39bb44bca9b4dc1aa2395bd22df not found: ID does not exist"
Jul 08 03:25:32.698179 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.698158    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66b449c4f7-sp8qb"]
Jul 08 03:25:32.704059 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:32.704042    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66b449c4f7-sp8qb"]
Jul 08 03:25:34.458819 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:34.458782    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="919205f6-7f4b-434c-9564-312f471eaf83" path="/var/lib/kubelet/pods/919205f6-7f4b-434c-9564-312f471eaf83/volumes"
Jul 08 03:25:36.707847 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:36.707817    2580 generic.go:356] "Generic (PLEG): container finished" podID="68111e66-8ce8-4eec-b76b-363147c018eb" containerID="261c86a3c844e6c6cdc4202239a65dd122205adb66cb4db2e8fdd4f8d9a575f4" exitCode=0
Jul 08 03:25:36.708346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:36.707895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp" event={"ID":"68111e66-8ce8-4eec-b76b-363147c018eb","Type":"ContainerDied","Data":"261c86a3c844e6c6cdc4202239a65dd122205adb66cb4db2e8fdd4f8d9a575f4"}
Jul 08 03:25:36.708346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:36.708326    2580 scope.go:117] "RemoveContainer" containerID="261c86a3c844e6c6cdc4202239a65dd122205adb66cb4db2e8fdd4f8d9a575f4"
Jul 08 03:25:37.713945 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:37.713911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-5lrfp" event={"ID":"68111e66-8ce8-4eec-b76b-363147c018eb","Type":"ContainerStarted","Data":"70ad84557d05d6bd851cd1419cb7732827f4c6be78867ccbb2c312b10cec5165"}
Jul 08 03:25:41.608788 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:41.608736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:41.665389 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:41.665355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:42.112409 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:42.112377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:42.116118 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:42.116097    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6b999598d-cxtrh"
Jul 08 03:25:47.990205 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:47.990164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:48.090968 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:48.090914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:48.156482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:48.156453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:25:58.299054 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:58.299020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:25:58.301439 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:58.301420    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b289c98c-c888-4672-9b09-03e12ae66c52-metrics-certs\") pod \"network-metrics-daemon-8cxvd\" (UID: \"b289c98c-c888-4672-9b09-03e12ae66c52\") " pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:25:58.555743 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:58.555669    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zb8ld\""
Jul 08 03:25:58.563601 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:58.563584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cxvd"
Jul 08 03:25:58.697275 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:58.697240    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8cxvd"]
Jul 08 03:25:58.702926 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:25:58.702888    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb289c98c_c888_4672_9b09_03e12ae66c52.slice/crio-6ef255cb35a2e8d070ee0a8bedae1120d9d4b4edf951c20e78209985bbd10a33 WatchSource:0}: Error finding container 6ef255cb35a2e8d070ee0a8bedae1120d9d4b4edf951c20e78209985bbd10a33: Status 404 returned error can't find the container with id 6ef255cb35a2e8d070ee0a8bedae1120d9d4b4edf951c20e78209985bbd10a33
Jul 08 03:25:58.803473 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:25:58.803438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cxvd" event={"ID":"b289c98c-c888-4672-9b09-03e12ae66c52","Type":"ContainerStarted","Data":"6ef255cb35a2e8d070ee0a8bedae1120d9d4b4edf951c20e78209985bbd10a33"}
Jul 08 03:26:00.815300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:00.815263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cxvd" event={"ID":"b289c98c-c888-4672-9b09-03e12ae66c52","Type":"ContainerStarted","Data":"e0a9886850c4ea860dda5ad7ede9cd43a2971ec124c854f143b9dd2b33bc3264"}
Jul 08 03:26:00.815759 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:00.815314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cxvd" event={"ID":"b289c98c-c888-4672-9b09-03e12ae66c52","Type":"ContainerStarted","Data":"96c84a74c2d5ab9997bcf9ee64aa0414cdb2933f046c1c40f5776bd36acd93e7"}
Jul 08 03:26:00.833910 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:00.833855    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8cxvd" podStartSLOduration=374.767726959 podStartE2EDuration="6m15.833837714s" totalImagesPullingTime="1.066110755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:26:00.832651729 +0000 UTC m=+376.874836397" watchObservedRunningTime="2026-07-08 03:26:00.833837714 +0000 UTC m=+376.876022382"
Jul 08 03:26:00.932431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:00.932400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:00.998110 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:00.998081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:03.924881 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:03.924839    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:26:03.945151 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:03.945117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:26:04.837524 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:04.837468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:04.846852 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:04.846823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 03:26:04.939018 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:04.938989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:13.479501 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:13.479459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:13.533734 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:13.533708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:23.783526 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:23.783445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:23.852132 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:23.852099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:25.389400 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:25.389355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:25.487586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:25.487551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:30.299711 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:26:30.299660    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z" podUID="5e7b6863-5540-4e0c-a79b-a725921fcd4e"
Jul 08 03:26:30.933764 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:30.933735    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:26:32.392294 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:32.392241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:26:32.394736 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:32.394704    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e7b6863-5540-4e0c-a79b-a725921fcd4e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-5h75z\" (UID: \"5e7b6863-5540-4e0c-a79b-a725921fcd4e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:26:32.437295 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:32.437272    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gk4k4\""
Jul 08 03:26:32.445102 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:32.445083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"
Jul 08 03:26:32.574516 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:32.574495    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-5h75z"]
Jul 08 03:26:32.576896 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:26:32.576872    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7b6863_5540_4e0c_a79b_a725921fcd4e.slice/crio-2878fcb6f5e500f259f1729aa5e5840a3bd7d7e8bfc6feb2fe79b85f2a8dbd8d WatchSource:0}: Error finding container 2878fcb6f5e500f259f1729aa5e5840a3bd7d7e8bfc6feb2fe79b85f2a8dbd8d: Status 404 returned error can't find the container with id 2878fcb6f5e500f259f1729aa5e5840a3bd7d7e8bfc6feb2fe79b85f2a8dbd8d
Jul 08 03:26:32.942566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:32.942528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z" event={"ID":"5e7b6863-5540-4e0c-a79b-a725921fcd4e","Type":"ContainerStarted","Data":"2878fcb6f5e500f259f1729aa5e5840a3bd7d7e8bfc6feb2fe79b85f2a8dbd8d"}
Jul 08 03:26:33.947407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:33.947317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z" event={"ID":"5e7b6863-5540-4e0c-a79b-a725921fcd4e","Type":"ContainerStarted","Data":"133e7ac18521556472792a681141705b1a2d597364645a820ca89793a0daa6bd"}
Jul 08 03:26:33.964505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:33.964456    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-5h75z" podStartSLOduration=383.878500552 podStartE2EDuration="6m24.964441845s" totalImagesPullingTime="1.085941293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:26:33.963353075 +0000 UTC m=+410.005537744" watchObservedRunningTime="2026-07-08 03:26:33.964441845 +0000 UTC m=+410.006626513"
Jul 08 03:26:34.067702 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:34.067667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:34.169537 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:34.169497    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:35.002106 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:35.002078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:35.058039 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:35.057997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:35.118051 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:35.118013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:41.028741 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:41.028706    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:41.084029 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:41.083981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:41.615211 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:41.615178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:41.713050 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:41.713013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:54.347895 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:54.347858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:54.402990 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:54.402935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:26:58.357757 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:26:58.357714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:00.556550 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:00.556515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:00.649232 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:00.649201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:18.716307 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:18.716219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:18.774840 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:18.774808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:22.268197 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:22.268147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:23.148645 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:23.148576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:26.466598 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:26.466568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:26.567553 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:26.567519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:27.523142 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:27.523109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:27.760832 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:27.760796    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:28.207241 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:28.207203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:28.375629 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:28.375593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:29.997303 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:29.997252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:30.593450 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:30.593410    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:34.088395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:34.088361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:34.187345 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:34.187301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:39.401601 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.401567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:39.522537 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522508    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f8jcz"]
Jul 08 03:27:39.522838 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522811    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026332f7-0884-4ea1-a23c-bf500d768aec" containerName="console"
Jul 08 03:27:39.522838 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522827    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="026332f7-0884-4ea1-a23c-bf500d768aec" containerName="console"
Jul 08 03:27:39.523077 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522868    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="919205f6-7f4b-434c-9564-312f471eaf83" containerName="registry"
Jul 08 03:27:39.523077 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522877    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="919205f6-7f4b-434c-9564-312f471eaf83" containerName="registry"
Jul 08 03:27:39.523077 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522983    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="026332f7-0884-4ea1-a23c-bf500d768aec" containerName="console"
Jul 08 03:27:39.523077 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.522998    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="919205f6-7f4b-434c-9564-312f471eaf83" containerName="registry"
Jul 08 03:27:39.529036 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.527217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.530111 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.530092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nk4rv\""
Jul 08 03:27:39.530245 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.530203    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 08 03:27:39.530901 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.530888    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 08 03:27:39.536538 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.536518    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f8jcz"]
Jul 08 03:27:39.658164 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.658083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d-bound-sa-token\") pod \"cert-manager-55b4d954d5-f8jcz\" (UID: \"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d\") " pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.658301 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.658181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltvq9\" (UniqueName: \"kubernetes.io/projected/0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d-kube-api-access-ltvq9\") pod \"cert-manager-55b4d954d5-f8jcz\" (UID: \"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d\") " pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.759273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.759233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d-bound-sa-token\") pod \"cert-manager-55b4d954d5-f8jcz\" (UID: \"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d\") " pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.759415 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.759343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltvq9\" (UniqueName: \"kubernetes.io/projected/0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d-kube-api-access-ltvq9\") pod \"cert-manager-55b4d954d5-f8jcz\" (UID: \"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d\") " pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.768294 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.768269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d-bound-sa-token\") pod \"cert-manager-55b4d954d5-f8jcz\" (UID: \"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d\") " pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.768495 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.768471    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltvq9\" (UniqueName: \"kubernetes.io/projected/0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d-kube-api-access-ltvq9\") pod \"cert-manager-55b4d954d5-f8jcz\" (UID: \"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d\") " pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.849084 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.849059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f8jcz"
Jul 08 03:27:39.981866 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:39.981669    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f8jcz"]
Jul 08 03:27:39.984767 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:27:39.984741    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ef8ada7_ad75_4d84_a5d1_f2114a9ec71d.slice/crio-26f80c75796410d03de03dcec6866965bcdc1567c8fd6a010dbf26d07eca7520 WatchSource:0}: Error finding container 26f80c75796410d03de03dcec6866965bcdc1567c8fd6a010dbf26d07eca7520: Status 404 returned error can't find the container with id 26f80c75796410d03de03dcec6866965bcdc1567c8fd6a010dbf26d07eca7520
Jul 08 03:27:40.223997 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:40.223898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f8jcz" event={"ID":"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d","Type":"ContainerStarted","Data":"26f80c75796410d03de03dcec6866965bcdc1567c8fd6a010dbf26d07eca7520"}
Jul 08 03:27:52.556991 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:52.556930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:52.654558 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:52.654524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:57.927477 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:57.927429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:58.028241 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:58.028210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:27:59.795597 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:27:59.795562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:02.325759 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:02.325716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f8jcz" event={"ID":"0ef8ada7-ad75-4d84-a5d1-f2114a9ec71d","Type":"ContainerStarted","Data":"a261a8df86b945f5d32d77399cc721a31d4142984d132cc6629d9f884cc060cd"}
Jul 08 03:28:02.345725 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:02.345650    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-f8jcz" podStartSLOduration=1.097597081 podStartE2EDuration="23.345628368s" totalImagesPullingTime="22.248031287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:02.344070998 +0000 UTC m=+498.386255667" watchObservedRunningTime="2026-07-08 03:28:02.345628368 +0000 UTC m=+498.387813086"
Jul 08 03:28:03.673135 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:03.673102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:03.776631 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:03.776594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:08.043989 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.043933    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:08.108876 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.108842    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"]
Jul 08 03:28:08.118226 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.118192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.122005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.121943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 08 03:28:08.122005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.121997    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8qpm9\""
Jul 08 03:28:08.122005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.121987    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 08 03:28:08.122236 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.122069    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:28:08.122236 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.121990    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 08 03:28:08.122236 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.122003    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 08 03:28:08.124262 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.124238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"]
Jul 08 03:28:08.151875 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.151843    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:08.194096 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.194065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-manager-config\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.194096 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.194095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-cert\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.194344 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.194114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-metrics-cert\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.194344 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.194255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqwn5\" (UniqueName: \"kubernetes.io/projected/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-kube-api-access-xqwn5\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.295391 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.295294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqwn5\" (UniqueName: \"kubernetes.io/projected/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-kube-api-access-xqwn5\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.295391 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.295369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-manager-config\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.295391 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.295389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-cert\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.295615 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.295404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-metrics-cert\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.296107 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.296068    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-manager-config\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.298091 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.298069    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-metrics-cert\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.298188 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.298103    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-cert\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.303718 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.303696    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqwn5\" (UniqueName: \"kubernetes.io/projected/f00a7d3c-ac59-4871-b4d8-7c200747c8ce-kube-api-access-xqwn5\") pod \"lws-controller-manager-d464d4856-sddcv\" (UID: \"f00a7d3c-ac59-4871-b4d8-7c200747c8ce\") " pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.428730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.428699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:08.562600 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:08.562574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"]
Jul 08 03:28:08.565527 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:28:08.565498    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00a7d3c_ac59_4871_b4d8_7c200747c8ce.slice/crio-04a78a66d467454597a1f535c97adbe08c40993a529119bf9b2091fde5c41d6f WatchSource:0}: Error finding container 04a78a66d467454597a1f535c97adbe08c40993a529119bf9b2091fde5c41d6f: Status 404 returned error can't find the container with id 04a78a66d467454597a1f535c97adbe08c40993a529119bf9b2091fde5c41d6f
Jul 08 03:28:09.358833 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:09.358795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv" event={"ID":"f00a7d3c-ac59-4871-b4d8-7c200747c8ce","Type":"ContainerStarted","Data":"04a78a66d467454597a1f535c97adbe08c40993a529119bf9b2091fde5c41d6f"}
Jul 08 03:28:11.369070 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:11.369031    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv" event={"ID":"f00a7d3c-ac59-4871-b4d8-7c200747c8ce","Type":"ContainerStarted","Data":"dfa47b1f218b0b1e7af376ac994471055f5ea18a90703e4c1e19397d5e00d244"}
Jul 08 03:28:11.369447 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:11.369159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:11.386082 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:11.386028    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv" podStartSLOduration=0.93034498 podStartE2EDuration="3.386013511s" totalImagesPullingTime="2.455668531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:11.38439792 +0000 UTC m=+507.426582589" watchObservedRunningTime="2026-07-08 03:28:11.386013511 +0000 UTC m=+507.428198180"
Jul 08 03:28:14.714478 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:14.714438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:14.813051 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:14.813020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:15.326323 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.326292    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:15.550795 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.550757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:15.648200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.648104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:15.687938 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.687904    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"]
Jul 08 03:28:15.691475 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.691453    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.694095 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.694055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 08 03:28:15.694095 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.694071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9xhxh\""
Jul 08 03:28:15.694540 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.694512    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 08 03:28:15.695466 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.695440    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 08 03:28:15.695466 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.695460    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 08 03:28:15.716608 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.716575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"]
Jul 08 03:28:15.761897 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.761859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g5pq\" (UniqueName: \"kubernetes.io/projected/d00b7e21-ef64-4492-8345-7ae2881cb101-kube-api-access-9g5pq\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.761897 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.761896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d00b7e21-ef64-4492-8345-7ae2881cb101-tmp\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.762156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.761981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d00b7e21-ef64-4492-8345-7ae2881cb101-manifests\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.762156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.762004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d00b7e21-ef64-4492-8345-7ae2881cb101-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.762156 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.762028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d00b7e21-ef64-4492-8345-7ae2881cb101-webhook-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.862745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.862696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g5pq\" (UniqueName: \"kubernetes.io/projected/d00b7e21-ef64-4492-8345-7ae2881cb101-kube-api-access-9g5pq\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.862745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.862741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d00b7e21-ef64-4492-8345-7ae2881cb101-tmp\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.863056 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.862877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d00b7e21-ef64-4492-8345-7ae2881cb101-manifests\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.863056 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.862923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d00b7e21-ef64-4492-8345-7ae2881cb101-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.863056 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.862996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d00b7e21-ef64-4492-8345-7ae2881cb101-webhook-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.863258 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.863241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d00b7e21-ef64-4492-8345-7ae2881cb101-manifests\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.865227 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.865196    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d00b7e21-ef64-4492-8345-7ae2881cb101-tmp\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.865487 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.865471    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d00b7e21-ef64-4492-8345-7ae2881cb101-webhook-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.865541 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.865515    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d00b7e21-ef64-4492-8345-7ae2881cb101-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.878596 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.878570    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:15.880497 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.880479    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g5pq\" (UniqueName: \"kubernetes.io/projected/d00b7e21-ef64-4492-8345-7ae2881cb101-kube-api-access-9g5pq\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx\" (UID: \"d00b7e21-ef64-4492-8345-7ae2881cb101\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:15.981339 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:15.981243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:16.002157 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.002125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:16.082556 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.082521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:16.146709 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.146670    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"]
Jul 08 03:28:16.149923 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:28:16.149897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00b7e21_ef64_4492_8345_7ae2881cb101.slice/crio-d8012f611a502cf60a8ab08a9b8f4400d8b45ae57b3b7c73ecd50c8b4e94d81b WatchSource:0}: Error finding container d8012f611a502cf60a8ab08a9b8f4400d8b45ae57b3b7c73ecd50c8b4e94d81b: Status 404 returned error can't find the container with id d8012f611a502cf60a8ab08a9b8f4400d8b45ae57b3b7c73ecd50c8b4e94d81b
Jul 08 03:28:16.150995 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.150829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:16.230904 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.230853    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:16.331119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.331084    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:16.390759 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:16.390725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx" event={"ID":"d00b7e21-ef64-4492-8345-7ae2881cb101","Type":"ContainerStarted","Data":"d8012f611a502cf60a8ab08a9b8f4400d8b45ae57b3b7c73ecd50c8b4e94d81b"}
Jul 08 03:28:20.411359 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:20.411322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx" event={"ID":"d00b7e21-ef64-4492-8345-7ae2881cb101","Type":"ContainerStarted","Data":"1929cf761cb9b8c715760b3d3b903d717548bfad051d7547e99ea5714747b332"}
Jul 08 03:28:21.417588 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:21.417546    2580 generic.go:356] "Generic (PLEG): container finished" podID="d00b7e21-ef64-4492-8345-7ae2881cb101" containerID="1929cf761cb9b8c715760b3d3b903d717548bfad051d7547e99ea5714747b332" exitCode=0
Jul 08 03:28:21.418062 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:21.417626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx" event={"ID":"d00b7e21-ef64-4492-8345-7ae2881cb101","Type":"ContainerDied","Data":"1929cf761cb9b8c715760b3d3b903d717548bfad051d7547e99ea5714747b332"}
Jul 08 03:28:22.375174 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:22.375142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d464d4856-sddcv"
Jul 08 03:28:22.428856 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:22.428808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx" event={"ID":"d00b7e21-ef64-4492-8345-7ae2881cb101","Type":"ContainerStarted","Data":"c5bb77b03c15687a2a76d59cf512e3c50e986b0dc2c76d7b669f5047fb4dfa4e"}
Jul 08 03:28:22.429360 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:22.429120    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:22.450776 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:22.450733    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx" podStartSLOduration=2.7828958569999998 podStartE2EDuration="7.450689792s" totalImagesPullingTime="3.640864862s" totalInitContainerRuntime="1.026929073s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:22.449304278 +0000 UTC m=+518.491488949" watchObservedRunningTime="2026-07-08 03:28:22.450689792 +0000 UTC m=+518.492874457"
Jul 08 03:28:33.435474 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:33.435445    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx"
Jul 08 03:28:34.926658 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:34.926615    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:35.111277 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.111247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:35.398682 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.398648    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:35.523167 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.523092    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"]
Jul 08 03:28:35.598989 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.598927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:35.681031 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.680933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"]
Jul 08 03:28:35.681182 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.681056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.684416 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.684390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 08 03:28:35.684671 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.684653    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jds27\""
Jul 08 03:28:35.685121 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.685104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 03:28:35.685724 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.685704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 08 03:28:35.685815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.685781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 03:28:35.841360 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.841327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80369dcb-9893-491f-bcd8-b6a7f595e19c-tmp\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.841531 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.841392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-577vd\" (UniqueName: \"kubernetes.io/projected/80369dcb-9893-491f-bcd8-b6a7f595e19c-kube-api-access-577vd\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.841531 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.841461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80369dcb-9893-491f-bcd8-b6a7f595e19c-tls-certs\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.859084 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.859050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:35.942933 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.942822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80369dcb-9893-491f-bcd8-b6a7f595e19c-tmp\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.943387 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.942969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-577vd\" (UniqueName: \"kubernetes.io/projected/80369dcb-9893-491f-bcd8-b6a7f595e19c-kube-api-access-577vd\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.943387 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.943022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80369dcb-9893-491f-bcd8-b6a7f595e19c-tls-certs\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.945277 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.945252    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80369dcb-9893-491f-bcd8-b6a7f595e19c-tmp\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.945487 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.945470    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80369dcb-9893-491f-bcd8-b6a7f595e19c-tls-certs\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.970337 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.970311    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-577vd\" (UniqueName: \"kubernetes.io/projected/80369dcb-9893-491f-bcd8-b6a7f595e19c-kube-api-access-577vd\") pod \"kube-auth-proxy-69556cfdfb-vb2xt\" (UID: \"80369dcb-9893-491f-bcd8-b6a7f595e19c\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.991200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.991174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"
Jul 08 03:28:35.994008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:35.993971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:36.120108 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:36.120075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:36.131308 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:36.131186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt"]
Jul 08 03:28:36.134131 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:28:36.134104    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80369dcb_9893_491f_bcd8_b6a7f595e19c.slice/crio-6939902fb40a8a5c234cc935dcd225cd5ce0a2baf84540176a2aacc21d1b2bf5 WatchSource:0}: Error finding container 6939902fb40a8a5c234cc935dcd225cd5ce0a2baf84540176a2aacc21d1b2bf5: Status 404 returned error can't find the container with id 6939902fb40a8a5c234cc935dcd225cd5ce0a2baf84540176a2aacc21d1b2bf5
Jul 08 03:28:36.291142 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:36.291039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:36.492463 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:36.492426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt" event={"ID":"80369dcb-9893-491f-bcd8-b6a7f595e19c","Type":"ContainerStarted","Data":"6939902fb40a8a5c234cc935dcd225cd5ce0a2baf84540176a2aacc21d1b2bf5"}
Jul 08 03:28:36.819173 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:36.819141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:37.069297 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:37.069203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:37.177101 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:37.177054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:38.204858 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:38.204820    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:39.529992 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:39.529941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt" event={"ID":"80369dcb-9893-491f-bcd8-b6a7f595e19c","Type":"ContainerStarted","Data":"21224eaac323008565702548b4fc9d49a21d0c1616198c6f750f3958bebeb65d"}
Jul 08 03:28:39.549097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:39.549049    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-vb2xt" podStartSLOduration=2.048480248 podStartE2EDuration="4.549033088s" totalImagesPullingTime="2.50055284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:39.547881877 +0000 UTC m=+535.590066547" watchObservedRunningTime="2026-07-08 03:28:39.549033088 +0000 UTC m=+535.591217737"
Jul 08 03:28:42.726398 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:42.726355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:42.822437 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:42.822407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:43.066398 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:43.066366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:43.381173 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:43.381068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:43.486895 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:43.486852    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:46.514535 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:46.514491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:46.645274 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:46.645238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:46.779083 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:46.778981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:46.995497 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:46.995433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:47.810446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:47.810402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:47.916330 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:47.916263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:48.036597 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:48.036563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:52.730918 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:52.730879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:28:54.470618 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:28:54.470582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:29:36.414884 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:36.414849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:29:36.506923 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:36.506885    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:29:36.613072 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:36.613034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:29:37.349108 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:37.349072    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:29:37.555319 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:37.555287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:29:44.362808 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:44.362783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:29:44.364061 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:44.364040    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:29:51.139815 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:29:51.139777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:05.404666 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:05.404634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:06.000687 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:06.000654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:07.039217 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:07.039171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:10.432045 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:10.432013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:13.610567 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:13.610530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:13.880525 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:13.880433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:16.271028 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:16.270981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:16.432698 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:16.432664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:16.598805 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:16.598772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:16.719630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:16.719595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:18.905085 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:18.905049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:21.420147 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:21.420108    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:21.522364 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:21.522332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:21.632930 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:21.632896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:21.758775 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:21.758690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:25.221898 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:25.221852    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:25.322634 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:25.322602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:27.290473 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:27.290437    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:27.479928 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:27.479894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:27.583413 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:27.583375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:29.270670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:29.270638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:29.370289 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:29.370249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:31.038349 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.038322    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"]
Jul 08 03:30:31.042000 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.041984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.044593 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.044565    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 08 03:30:31.044781 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.044764    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 03:30:31.045401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.045383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7mzxj\""
Jul 08 03:30:31.045495 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.045408    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 08 03:30:31.045495 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.045435    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 03:30:31.050346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.050327    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"]
Jul 08 03:30:31.050922 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.050891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:31.147300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.147266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.147469 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.147372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.147469 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.147394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67l44\" (UniqueName: \"kubernetes.io/projected/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-kube-api-access-67l44\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.248576 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.248527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.248576 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.248580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67l44\" (UniqueName: \"kubernetes.io/projected/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-kube-api-access-67l44\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.248781 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.248619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.248781 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:30:31.248738    2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 08 03:30:31.248846 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:30:31.248803    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-plugin-serving-cert podName:b6efde98-36d5-425d-ac7a-d6ab4ff33b86 nodeName:}" failed. No retries permitted until 2026-07-08 03:30:31.748782851 +0000 UTC m=+647.790967501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-7fw95" (UID: "b6efde98-36d5-425d-ac7a-d6ab4ff33b86") : secret "plugin-serving-cert" not found
Jul 08 03:30:31.249250 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.249232    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.262522 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.262500    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67l44\" (UniqueName: \"kubernetes.io/projected/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-kube-api-access-67l44\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.481036 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.481000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:31.752889 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.752812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.755383 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.755349    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6efde98-36d5-425d-ac7a-d6ab4ff33b86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7fw95\" (UID: \"b6efde98-36d5-425d-ac7a-d6ab4ff33b86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:31.952192 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:31.952167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"
Jul 08 03:30:32.083300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:32.083241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95"]
Jul 08 03:30:32.085771 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:30:32.085743    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6efde98_36d5_425d_ac7a_d6ab4ff33b86.slice/crio-ac4693dfcb236507088c9016c4d033da0bbffc51eb978142e031db1c0d8e580f WatchSource:0}: Error finding container ac4693dfcb236507088c9016c4d033da0bbffc51eb978142e031db1c0d8e580f: Status 404 returned error can't find the container with id ac4693dfcb236507088c9016c4d033da0bbffc51eb978142e031db1c0d8e580f
Jul 08 03:30:32.087487 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:32.087471    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:30:32.772166 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:32.772126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:32.872382 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:32.872352    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:33.046559 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:33.046525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95" event={"ID":"b6efde98-36d5-425d-ac7a-d6ab4ff33b86","Type":"ContainerStarted","Data":"ac4693dfcb236507088c9016c4d033da0bbffc51eb978142e031db1c0d8e580f"}
Jul 08 03:30:39.407306 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:39.407258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:39.507579 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:39.507540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:40.798886 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:40.798843    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:40.903739 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:40.903699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:41.472233 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:41.472199    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:41.590035 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:41.589999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:42.749598 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.749565    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"]
Jul 08 03:30:42.780685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.780651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"]
Jul 08 03:30:42.780870 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.780785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:42.783146 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.783117    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d2xbg\""
Jul 08 03:30:42.855649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.855597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvrh7\" (UniqueName: \"kubernetes.io/projected/4f117951-484b-4646-b02c-6383ed728d6d-kube-api-access-vvrh7\") pod \"kuadrant-operator-controller-manager-55467d9f7-knq7m\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:42.855835 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.855799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f117951-484b-4646-b02c-6383ed728d6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-knq7m\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:42.956626 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.956584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f117951-484b-4646-b02c-6383ed728d6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-knq7m\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:42.956794 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.956643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrh7\" (UniqueName: \"kubernetes.io/projected/4f117951-484b-4646-b02c-6383ed728d6d-kube-api-access-vvrh7\") pod \"kuadrant-operator-controller-manager-55467d9f7-knq7m\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:42.957018 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.956994    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f117951-484b-4646-b02c-6383ed728d6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-knq7m\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:42.966514 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:42.966484    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvrh7\" (UniqueName: \"kubernetes.io/projected/4f117951-484b-4646-b02c-6383ed728d6d-kube-api-access-vvrh7\") pod \"kuadrant-operator-controller-manager-55467d9f7-knq7m\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:43.095313 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:43.095266    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:30:44.426071 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:44.426036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:44.518423 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:44.518386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:49.002906 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:49.002846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:49.109822 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:49.109784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:52.407249 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:52.407211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:52.507090 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:52.507045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:30:54.434423 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:54.434394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"]
Jul 08 03:30:54.436302 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:30:54.436278    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f117951_484b_4646_b02c_6383ed728d6d.slice/crio-21a4fb27fe97ccaf28a289063e22c3e28c958b48873307329d82d1dead36c758 WatchSource:0}: Error finding container 21a4fb27fe97ccaf28a289063e22c3e28c958b48873307329d82d1dead36c758: Status 404 returned error can't find the container with id 21a4fb27fe97ccaf28a289063e22c3e28c958b48873307329d82d1dead36c758
Jul 08 03:30:55.167850 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:55.167809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95" event={"ID":"b6efde98-36d5-425d-ac7a-d6ab4ff33b86","Type":"ContainerStarted","Data":"a05d22c381c1ac12007a4db44d68ff4369cd88cc437f078eee758c8c7a4af066"}
Jul 08 03:30:55.169016 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:55.168993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m" event={"ID":"4f117951-484b-4646-b02c-6383ed728d6d","Type":"ContainerStarted","Data":"21a4fb27fe97ccaf28a289063e22c3e28c958b48873307329d82d1dead36c758"}
Jul 08 03:30:55.199745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:30:55.199699    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7fw95" podStartSLOduration=1.578727621 podStartE2EDuration="24.199684067s" totalImagesPullingTime="22.620956446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:55.196379892 +0000 UTC m=+671.238564562" watchObservedRunningTime="2026-07-08 03:30:55.199684067 +0000 UTC m=+671.241868736"
Jul 08 03:31:01.203351 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:01.203314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m" event={"ID":"4f117951-484b-4646-b02c-6383ed728d6d","Type":"ContainerStarted","Data":"d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4"}
Jul 08 03:31:01.203853 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:01.203537    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:31:01.224589 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:01.224539    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m" podStartSLOduration=13.075467181 podStartE2EDuration="19.22452457s" totalImagesPullingTime="6.149057389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:01.223473734 +0000 UTC m=+677.265658404" watchObservedRunningTime="2026-07-08 03:31:01.22452457 +0000 UTC m=+677.266709239"
Jul 08 03:31:10.233187 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:10.233148    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:12.209648 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:12.209607    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:31:24.429164 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:24.429128    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"]
Jul 08 03:31:24.429623 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:24.429425    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m" podUID="4f117951-484b-4646-b02c-6383ed728d6d" containerName="manager" containerID="cri-o://d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4" gracePeriod=10
Jul 08 03:31:25.213728 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.213697    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:31:25.324063 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.323982    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f117951-484b-4646-b02c-6383ed728d6d" containerID="d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4" exitCode=0
Jul 08 03:31:25.324063 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.324021    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"
Jul 08 03:31:25.324238 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.324050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m" event={"ID":"4f117951-484b-4646-b02c-6383ed728d6d","Type":"ContainerDied","Data":"d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4"}
Jul 08 03:31:25.324238 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.324115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m" event={"ID":"4f117951-484b-4646-b02c-6383ed728d6d","Type":"ContainerDied","Data":"21a4fb27fe97ccaf28a289063e22c3e28c958b48873307329d82d1dead36c758"}
Jul 08 03:31:25.324238 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.324145    2580 scope.go:117] "RemoveContainer" containerID="d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4"
Jul 08 03:31:25.333045 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.333027    2580 scope.go:117] "RemoveContainer" containerID="d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4"
Jul 08 03:31:25.333307 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:31:25.333292    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4\": container with ID starting with d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4 not found: ID does not exist" containerID="d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4"
Jul 08 03:31:25.333367 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.333314    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4"} err="failed to get container status \"d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4\": rpc error: code = NotFound desc = could not find container \"d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4\": container with ID starting with d3a6771cad41aff83b824e97e235bbecb64095519978225520afc11db68658e4 not found: ID does not exist"
Jul 08 03:31:25.340202 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.340183    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f117951-484b-4646-b02c-6383ed728d6d-extensions-socket-volume\") pod \"4f117951-484b-4646-b02c-6383ed728d6d\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") "
Jul 08 03:31:25.340330 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.340226    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvrh7\" (UniqueName: \"kubernetes.io/projected/4f117951-484b-4646-b02c-6383ed728d6d-kube-api-access-vvrh7\") pod \"4f117951-484b-4646-b02c-6383ed728d6d\" (UID: \"4f117951-484b-4646-b02c-6383ed728d6d\") "
Jul 08 03:31:25.340659 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.340637    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f117951-484b-4646-b02c-6383ed728d6d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f117951-484b-4646-b02c-6383ed728d6d" (UID: "4f117951-484b-4646-b02c-6383ed728d6d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:31:25.342574 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.342548    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f117951-484b-4646-b02c-6383ed728d6d-kube-api-access-vvrh7" (OuterVolumeSpecName: "kube-api-access-vvrh7") pod "4f117951-484b-4646-b02c-6383ed728d6d" (UID: "4f117951-484b-4646-b02c-6383ed728d6d"). InnerVolumeSpecName "kube-api-access-vvrh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:31:25.441216 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.441181    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvrh7\" (UniqueName: \"kubernetes.io/projected/4f117951-484b-4646-b02c-6383ed728d6d-kube-api-access-vvrh7\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:31:25.441216 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.441208    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f117951-484b-4646-b02c-6383ed728d6d-extensions-socket-volume\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:31:25.649193 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.649156    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"]
Jul 08 03:31:25.655531 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:25.655507    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-knq7m"]
Jul 08 03:31:26.457348 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:26.457316    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f117951-484b-4646-b02c-6383ed728d6d" path="/var/lib/kubelet/pods/4f117951-484b-4646-b02c-6383ed728d6d/volumes"
Jul 08 03:31:28.558679 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:28.558644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:28.771713 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:28.771646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:32.632769 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:32.632731    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:32.739676 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:32.739647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:32.803776 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:32.803747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:33.091047 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.091014    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:31:33.091362 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.091351    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f117951-484b-4646-b02c-6383ed728d6d" containerName="manager"
Jul 08 03:31:33.091405 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.091364    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f117951-484b-4646-b02c-6383ed728d6d" containerName="manager"
Jul 08 03:31:33.091439 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.091425    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f117951-484b-4646-b02c-6383ed728d6d" containerName="manager"
Jul 08 03:31:33.105300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.105271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:31:33.105451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.105379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.107501 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.107468    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 08 03:31:33.117597 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.117575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:31:33.208538 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.208507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q427v\" (UniqueName: \"kubernetes.io/projected/7f371001-c979-47f4-a621-521f2ecb30b9-kube-api-access-q427v\") pod \"limitador-limitador-54d6d988d5-wfn9v\" (UID: \"7f371001-c979-47f4-a621-521f2ecb30b9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.208681 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.208555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f371001-c979-47f4-a621-521f2ecb30b9-config-file\") pod \"limitador-limitador-54d6d988d5-wfn9v\" (UID: \"7f371001-c979-47f4-a621-521f2ecb30b9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.309282 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.309233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f371001-c979-47f4-a621-521f2ecb30b9-config-file\") pod \"limitador-limitador-54d6d988d5-wfn9v\" (UID: \"7f371001-c979-47f4-a621-521f2ecb30b9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.309433 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.309369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q427v\" (UniqueName: \"kubernetes.io/projected/7f371001-c979-47f4-a621-521f2ecb30b9-kube-api-access-q427v\") pod \"limitador-limitador-54d6d988d5-wfn9v\" (UID: \"7f371001-c979-47f4-a621-521f2ecb30b9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.309990 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.309946    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f371001-c979-47f4-a621-521f2ecb30b9-config-file\") pod \"limitador-limitador-54d6d988d5-wfn9v\" (UID: \"7f371001-c979-47f4-a621-521f2ecb30b9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.317073 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.317053    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q427v\" (UniqueName: \"kubernetes.io/projected/7f371001-c979-47f4-a621-521f2ecb30b9-kube-api-access-q427v\") pod \"limitador-limitador-54d6d988d5-wfn9v\" (UID: \"7f371001-c979-47f4-a621-521f2ecb30b9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.416095 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.416028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:33.472683 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.472617    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:33.535610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.535576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:31:33.548925 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:33.548905    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:31:33.551623 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:31:33.551594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f371001_c979_47f4_a621_521f2ecb30b9.slice/crio-d7d3c3ca77ecfbf732b19fb61cbbff48e8e4df9efcd284af5759e18506b53a5b WatchSource:0}: Error finding container d7d3c3ca77ecfbf732b19fb61cbbff48e8e4df9efcd284af5759e18506b53a5b: Status 404 returned error can't find the container with id d7d3c3ca77ecfbf732b19fb61cbbff48e8e4df9efcd284af5759e18506b53a5b
Jul 08 03:31:34.369505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:34.369463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v" event={"ID":"7f371001-c979-47f4-a621-521f2ecb30b9","Type":"ContainerStarted","Data":"d7d3c3ca77ecfbf732b19fb61cbbff48e8e4df9efcd284af5759e18506b53a5b"}
Jul 08 03:31:36.381346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:36.381301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v" event={"ID":"7f371001-c979-47f4-a621-521f2ecb30b9","Type":"ContainerStarted","Data":"58fe1c96a3a0fd0f5e98fb2c0ad214c328de31bc7ccca54d329e0b9e50fa1fb9"}
Jul 08 03:31:36.381743 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:36.381360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:36.397883 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:36.397843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v" podStartSLOduration=1.037527708 podStartE2EDuration="3.397829022s" totalImagesPullingTime="2.360301314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:36.397334385 +0000 UTC m=+712.439519053" watchObservedRunningTime="2026-07-08 03:31:36.397829022 +0000 UTC m=+712.440013691"
Jul 08 03:31:47.387468 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:47.387434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"
Jul 08 03:31:51.922394 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:31:51.922362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:04.973753 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:04.973720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:05.253076 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:05.252982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:11.924355 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:11.924317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:12.006296 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:12.006266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:12.087080 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:12.087049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:12.144407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:12.144369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:14.701681 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:14.701635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:14.806578 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:14.806541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:20.228627 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:20.228592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:46.120205 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:46.120163    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:46.660192 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:46.660164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:47.101054 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.101024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:47.206296 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.206255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:47.241552 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.241516    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qx2xb"]
Jul 08 03:32:47.244974 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.244938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.248369 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.248345    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 08 03:32:47.261249 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.261222    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qx2xb"]
Jul 08 03:32:47.303796 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.303760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:32:47.315597 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.315566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:47.329776 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.329753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96mhk\" (UniqueName: \"kubernetes.io/projected/646410fa-87db-4499-b8df-86c468fd97a7-kube-api-access-96mhk\") pod \"payload-pre-processing-7964457696-qx2xb\" (UID: \"646410fa-87db-4499-b8df-86c468fd97a7\") " pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.329896 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.329877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/646410fa-87db-4499-b8df-86c468fd97a7-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qx2xb\" (UID: \"646410fa-87db-4499-b8df-86c468fd97a7\") " pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.430436 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.430364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/646410fa-87db-4499-b8df-86c468fd97a7-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qx2xb\" (UID: \"646410fa-87db-4499-b8df-86c468fd97a7\") " pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.430436 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.430413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96mhk\" (UniqueName: \"kubernetes.io/projected/646410fa-87db-4499-b8df-86c468fd97a7-kube-api-access-96mhk\") pod \"payload-pre-processing-7964457696-qx2xb\" (UID: \"646410fa-87db-4499-b8df-86c468fd97a7\") " pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.431270 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.431248    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/646410fa-87db-4499-b8df-86c468fd97a7-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qx2xb\" (UID: \"646410fa-87db-4499-b8df-86c468fd97a7\") " pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.438966 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.438925    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96mhk\" (UniqueName: \"kubernetes.io/projected/646410fa-87db-4499-b8df-86c468fd97a7-kube-api-access-96mhk\") pod \"payload-pre-processing-7964457696-qx2xb\" (UID: \"646410fa-87db-4499-b8df-86c468fd97a7\") " pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.555640 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.555608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:47.686453 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.686423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qx2xb"]
Jul 08 03:32:47.688340 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:32:47.688306    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646410fa_87db_4499_b8df_86c468fd97a7.slice/crio-7c64bc42a3b722af8050b5e5b1506c2d668694070ea53ac3267d9e66efff1f85 WatchSource:0}: Error finding container 7c64bc42a3b722af8050b5e5b1506c2d668694070ea53ac3267d9e66efff1f85: Status 404 returned error can't find the container with id 7c64bc42a3b722af8050b5e5b1506c2d668694070ea53ac3267d9e66efff1f85
Jul 08 03:32:47.704926 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:47.704895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb" event={"ID":"646410fa-87db-4499-b8df-86c468fd97a7","Type":"ContainerStarted","Data":"7c64bc42a3b722af8050b5e5b1506c2d668694070ea53ac3267d9e66efff1f85"}
Jul 08 03:32:49.716435 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:49.716393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb" event={"ID":"646410fa-87db-4499-b8df-86c468fd97a7","Type":"ContainerStarted","Data":"5d1d7a75f61145eb3cb1ea534257b4ae52628442561b92800266b553f27c239b"}
Jul 08 03:32:49.716846 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:49.716534    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:32:49.734678 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:49.734607    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb" podStartSLOduration=1.288329259 podStartE2EDuration="2.734588016s" totalImagesPullingTime="1.446258757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:49.733944878 +0000 UTC m=+785.776129551" watchObservedRunningTime="2026-07-08 03:32:49.734588016 +0000 UTC m=+785.776772687"
Jul 08 03:32:57.367759 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:57.367714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:32:57.467620 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:32:57.467586    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:33:00.724040 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:00.724001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-qx2xb"
Jul 08 03:33:28.979916 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:28.979879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:33:29.452791 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:29.452742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:33:43.158269 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:43.158227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:33:43.831304 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:43.831265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:33:53.148558 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:53.148526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:33:53.250166 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:33:53.250131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:01.336176 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.336138    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"]
Jul 08 03:34:01.341154 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.341131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.344441 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.344420    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tb89x\""
Jul 08 03:34:01.344554 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.344463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 08 03:34:01.344613 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.344577    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 08 03:34:01.344695 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.344681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 08 03:34:01.348418 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.348011    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"]
Jul 08 03:34:01.357089 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.357065    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:01.486459 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.486429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.486649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.486472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b48757e7-36de-4df7-a603-b3ab1354a9ca-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.486649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.486510    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.486649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.486525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.486649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.486579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7ktw\" (UniqueName: \"kubernetes.io/projected/b48757e7-36de-4df7-a603-b3ab1354a9ca-kube-api-access-m7ktw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.486649 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.486622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587345 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587345 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b48757e7-36de-4df7-a603-b3ab1354a9ca-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587565 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587565 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587565 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7ktw\" (UniqueName: \"kubernetes.io/projected/b48757e7-36de-4df7-a603-b3ab1354a9ca-kube-api-access-m7ktw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587565 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587700 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587679    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587754 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.587810 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.587789    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.589800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.589778    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b48757e7-36de-4df7-a603-b3ab1354a9ca-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.590132 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.590114    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b48757e7-36de-4df7-a603-b3ab1354a9ca-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.595684 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.595664    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7ktw\" (UniqueName: \"kubernetes.io/projected/b48757e7-36de-4df7-a603-b3ab1354a9ca-kube-api-access-m7ktw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kjh9f\" (UID: \"b48757e7-36de-4df7-a603-b3ab1354a9ca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.651845 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.651815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:01.781688 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:01.781659    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"]
Jul 08 03:34:01.783063 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:34:01.783033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb48757e7_36de_4df7_a603_b3ab1354a9ca.slice/crio-63b70ef1369a2c7f1204c6380c3cb8019b7b8ca50e38e1dfe0728aeb3a293abb WatchSource:0}: Error finding container 63b70ef1369a2c7f1204c6380c3cb8019b7b8ca50e38e1dfe0728aeb3a293abb: Status 404 returned error can't find the container with id 63b70ef1369a2c7f1204c6380c3cb8019b7b8ca50e38e1dfe0728aeb3a293abb
Jul 08 03:34:02.059805 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:02.059759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f" event={"ID":"b48757e7-36de-4df7-a603-b3ab1354a9ca","Type":"ContainerStarted","Data":"63b70ef1369a2c7f1204c6380c3cb8019b7b8ca50e38e1dfe0728aeb3a293abb"}
Jul 08 03:34:02.436270 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:02.436178    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:34:03.067259 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:03.067167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f" event={"ID":"b48757e7-36de-4df7-a603-b3ab1354a9ca","Type":"ContainerStarted","Data":"95e7568d649a4927ccb90d298005ade2c22df22753c8c60e4acd5977b0755ba1"}
Jul 08 03:34:03.085792 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:03.085745    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f" podStartSLOduration=1.057362075 podStartE2EDuration="2.085728682s" totalImagesPullingTime="1.028366607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:34:03.085240526 +0000 UTC m=+859.127425195" watchObservedRunningTime="2026-07-08 03:34:03.085728682 +0000 UTC m=+859.127913355"
Jul 08 03:34:11.652278 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:11.652242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:11.652747 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:11.652319    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:11.664403 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:11.664373    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:12.121009 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:12.120979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kjh9f"
Jul 08 03:34:20.828006 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:20.827900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:22.240113 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:22.240081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:34:27.269969 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:27.269912    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:29.538514 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:29.538482    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:34:37.347355 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.347262    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"]
Jul 08 03:34:37.353401 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.353378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.356478 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.356460    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 08 03:34:37.366666 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.366643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"]
Jul 08 03:34:37.374592 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.374565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:37.393615 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.393594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m92wx\" (UniqueName: \"kubernetes.io/projected/be89fffc-feb6-4496-ac2f-b23663f7fa49-kube-api-access-m92wx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.393708 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.393671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.393767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.393705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.393767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.393734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.393874 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.393807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be89fffc-feb6-4496-ac2f-b23663f7fa49-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.393874 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.393842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495126 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be89fffc-feb6-4496-ac2f-b23663f7fa49-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m92wx\" (UniqueName: \"kubernetes.io/projected/be89fffc-feb6-4496-ac2f-b23663f7fa49-kube-api-access-m92wx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495539 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495511    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495615 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495591    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.495670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.495651    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.497526 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.497502    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be89fffc-feb6-4496-ac2f-b23663f7fa49-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.497936 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.497916    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be89fffc-feb6-4496-ac2f-b23663f7fa49-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.503286 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.503262    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m92wx\" (UniqueName: \"kubernetes.io/projected/be89fffc-feb6-4496-ac2f-b23663f7fa49-kube-api-access-m92wx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf\" (UID: \"be89fffc-feb6-4496-ac2f-b23663f7fa49\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.679982 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.679876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:37.814994 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:37.814964    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"]
Jul 08 03:34:37.816802 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:34:37.816773    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe89fffc_feb6_4496_ac2f_b23663f7fa49.slice/crio-aa28b92be1905b49418ecc62e888ff74543f4284b76f2c64551397ea72a90c70 WatchSource:0}: Error finding container aa28b92be1905b49418ecc62e888ff74543f4284b76f2c64551397ea72a90c70: Status 404 returned error can't find the container with id aa28b92be1905b49418ecc62e888ff74543f4284b76f2c64551397ea72a90c70
Jul 08 03:34:38.231849 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:38.231815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf" event={"ID":"be89fffc-feb6-4496-ac2f-b23663f7fa49","Type":"ContainerStarted","Data":"e9d7c0672e154926257d2073da7a06d9513b12a6b5f5978b3b474c8be4322af1"}
Jul 08 03:34:38.232047 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:38.231861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf" event={"ID":"be89fffc-feb6-4496-ac2f-b23663f7fa49","Type":"ContainerStarted","Data":"aa28b92be1905b49418ecc62e888ff74543f4284b76f2c64551397ea72a90c70"}
Jul 08 03:34:38.252910 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:38.252828    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf" podStartSLOduration=1.079186837 podStartE2EDuration="1.252812605s" totalImagesPullingTime="173.625768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:34:38.251836812 +0000 UTC m=+894.294021482" watchObservedRunningTime="2026-07-08 03:34:38.252812605 +0000 UTC m=+894.294997273"
Jul 08 03:34:39.946557 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:39.946521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:34:44.391980 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:44.391936    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:34:44.394503 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:44.394482    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:34:46.675409 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:46.675380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:46.776917 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:46.776883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:34:47.680116 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:47.680079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:47.680581 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:47.680135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:47.692703 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:47.692672    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:48.289676 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:48.289646    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf"
Jul 08 03:34:58.601107 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:34:58.601074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:14.497139 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:14.497104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:25.802732 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:25.802694    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:25.858115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:25.858079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:45.104965 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:45.104921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:45.200023 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:45.199985    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:46.193926 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:46.193889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:35:46.294823 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:35:46.294788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:36:12.443270 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:12.443193    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:36:22.746812 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:22.746770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:36:33.788641 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:33.788607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:36:43.747539 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:43.747504    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:36:53.839787 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:53.839747    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:36:56.251251 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:56.251217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:36:58.140451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:36:58.140417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:20.156663 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:20.156618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:37:33.340876 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:33.340840    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:37:36.619795 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:36.619712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:36.719157 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:36.719115    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:36.843458 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:36.843424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:42.729340 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:42.729300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:42.833864 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:42.833835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:42.929441 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:42.929406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:43.064070 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:43.064034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:48.861821 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:48.861782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:37:48.967855 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:37:48.967817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:38:05.443586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:05.443552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:38:15.136115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:15.136077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:38:23.545173 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:23.545132    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:38:26.018212 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:26.018179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:38:32.537005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:32.536975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:38:44.141893 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:44.141859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:38:55.137773 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:38:55.137738    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:39:06.956323 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:39:06.956247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:39:16.735161 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:39:16.735126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:39:38.336007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:39:38.335968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:39:44.420740 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:39:44.420709    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:39:44.424067 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:39:44.424049    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:39:48.931666 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:39:48.931632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:40:10.640323 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:40:10.640281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:40:18.302200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:40:18.302157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:40:21.234213 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:40:21.234176    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:40:37.447545 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:40:37.447461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:40:46.941824 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:40:46.941786    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:40:56.244219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:40:56.244183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:41:05.245834 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:41:05.245795    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:41:14.537512 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:41:14.537478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:41:33.968709 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:41:33.968668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:42:07.057395 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:42:07.057303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:42:07.117857 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:42:07.117825    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:43:05.736122 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:43:05.736083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:43:16.836248 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:43:16.836211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:43:25.339282 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:43:25.339241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:43:37.244467 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:43:37.244377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:43:48.541836 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:43:48.541803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:43:59.737397 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:43:59.737364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:44:08.334928 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:08.334891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:44:19.738140 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:19.738106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:44:30.140289 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:30.140253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:44:38.633295 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:38.633256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:44:44.451529 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:44.451497    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:44:44.454692 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:44.454670    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:44:51.739793 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:44:51.739758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:45:09.242618 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:45:09.242534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:45:18.738770 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:45:18.738733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:45:28.307571 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:45:28.307536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:45:39.341121 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:45:39.341083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:45:46.937214 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:45:46.937182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:45:58.537132 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:45:58.537087    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:46:09.839911 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:46:09.839879    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:46:19.340422 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:46:19.340389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:46:29.543807 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:46:29.543774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:46:38.737365 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:46:38.737285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:46:47.741864 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:46:47.741828    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:47:00.745718 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:47:00.745683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:47:10.243165 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:47:10.243129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:47:19.939752 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:47:19.939715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:47:29.740434 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:47:29.740391    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:47:40.746127 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:47:40.746092    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:47:50.043431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:47:50.043395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:01.336548 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:01.336510    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:12.243049 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:12.242967    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:21.539568 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:21.539530    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:25.224768 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:25.224732    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:32.077856 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:32.077819    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:34.535038 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:34.535007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:40.340630 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:40.340597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:43.533031 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:43.532995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:45.411560 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:45.411525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:48.158973 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:48.158909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:48.256730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:48.256696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:48.604341 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:48.604309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:54.408445 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:54.408411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:54.508418 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:54.508387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:54.610811 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:54.610777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:54.645345 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:54.645309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:48:54.671836 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:54.671766    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:55.792671 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:55.792635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:55.876386 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:55.876357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:55.940252 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:55.940222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:56.041511 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:56.041479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:56.359583 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:56.359545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:56.459634 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:56.459600    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:56.831784 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:56.831753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:48:58.294309 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:48:58.294272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:00.784273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:00.784238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:00.965333 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:00.965296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:01.065443 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:01.065419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:01.126963 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:01.126930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:04.511120 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:04.511093    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:06.824210 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:06.824180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:06.890764 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:06.890737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:06.997583 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:06.997553    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"]
Jul 08 03:49:07.001768 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.001747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.004436 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.004239    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 03:49:07.006435 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.005068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a129cbbc-openshift-default-dockercfg-7v2xz\""
Jul 08 03:49:07.021225 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.021195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"]
Jul 08 03:49:07.052332 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.052309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:07.119170 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.119092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:07.151391 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-certs\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151391 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-socket\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151600 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istiod-ca-cert\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151600 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-envoy\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151600 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-token\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151771 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-data\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151771 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb57d\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-kube-api-access-xb57d\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151771 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-podinfo\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.151771 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.151713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-credential-socket\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252473 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-envoy\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252606 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-token\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252606 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-data\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252606 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb57d\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-kube-api-access-xb57d\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252606 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-podinfo\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252606 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-credential-socket\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252863 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-certs\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252863 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-socket\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.252863 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.252660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istiod-ca-cert\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.253066 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.253010    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-data\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.253066 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.253046    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-socket\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.253169 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.253072    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-credential-socket\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.253169 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.253142    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-certs\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.253438 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.253423    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istiod-ca-cert\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.254980 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.254943    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-envoy\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.255091 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.255074    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-podinfo\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.260613 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.260589    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb57d\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-kube-api-access-xb57d\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.260715 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.260693    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-token\") pod \"e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") " pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.316300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.316271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:07.451574 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.451551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"]
Jul 08 03:49:07.453521 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:49:07.453492    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1d4767_1cfc_4fe9_9a83_2118145f6545.slice/crio-bfc3b8ca6abaedc78252f9f4501f14bd90a22a696f240b52a0daa1da1298aa8b WatchSource:0}: Error finding container bfc3b8ca6abaedc78252f9f4501f14bd90a22a696f240b52a0daa1da1298aa8b: Status 404 returned error can't find the container with id bfc3b8ca6abaedc78252f9f4501f14bd90a22a696f240b52a0daa1da1298aa8b
Jul 08 03:49:07.455489 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:07.455473    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:49:08.206972 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.206923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:08.292022 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.291996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:08.354646 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.354608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" event={"ID":"2b1d4767-1cfc-4fe9-9a83-2118145f6545","Type":"ContainerStarted","Data":"bfc3b8ca6abaedc78252f9f4501f14bd90a22a696f240b52a0daa1da1298aa8b"}
Jul 08 03:49:08.356524 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.356493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:08.457880 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.457767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:08.767042 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.766932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:08.867654 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:08.867619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:10.296131 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:10.296095    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:10.333458 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:10.333427    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 03:49:10.333536 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:10.333495    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 03:49:10.333536 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:10.333525    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 03:49:10.543691 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:10.543662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:49:11.374655 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:11.374615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" event={"ID":"2b1d4767-1cfc-4fe9-9a83-2118145f6545","Type":"ContainerStarted","Data":"017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862"}
Jul 08 03:49:11.394109 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:11.394048    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podStartSLOduration=2.516489881 podStartE2EDuration="5.394024917s" totalImagesPullingTime="2.877535036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:49:11.393690114 +0000 UTC m=+1767.435874779" watchObservedRunningTime="2026-07-08 03:49:11.394024917 +0000 UTC m=+1767.436209586"
Jul 08 03:49:12.316704 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:12.316665    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:13.187941 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:13.187905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:13.331306 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:13.331257    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:49:13.331647 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:13.331606    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:49:13.495382 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:13.495296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:14.333218 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:14.333158    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:49:14.334501 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:14.333833    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:49:15.336523 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:15.336064    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:49:15.336523 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:15.336160    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:49:16.377160 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:16.377105    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:49:16.377659 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:16.377182    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:49:17.418115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:17.418068    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:49:17.418586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:17.418137    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:49:18.451451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:18.451309    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:49:18.451451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:18.451369    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:49:18.451921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:18.451765    2580 patch_prober.go:28] interesting pod/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 08 03:49:18.451921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:18.451812    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 08 03:49:19.264694 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.264653    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"]
Jul 08 03:49:19.264921 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.264895    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy" containerID="cri-o://017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862" gracePeriod=30
Jul 08 03:49:19.290790 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.290764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:19.386280 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.386251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:19.400818 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.400793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:19.415024 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.414992    2580 generic.go:356] "Generic (PLEG): container finished" podID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerID="017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862" exitCode=0
Jul 08 03:49:19.415182 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.415040    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"
Jul 08 03:49:19.415182 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.415081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" event={"ID":"2b1d4767-1cfc-4fe9-9a83-2118145f6545","Type":"ContainerDied","Data":"017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862"}
Jul 08 03:49:19.415182 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.415137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv" event={"ID":"2b1d4767-1cfc-4fe9-9a83-2118145f6545","Type":"ContainerDied","Data":"bfc3b8ca6abaedc78252f9f4501f14bd90a22a696f240b52a0daa1da1298aa8b"}
Jul 08 03:49:19.415182 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.415159    2580 scope.go:117] "RemoveContainer" containerID="017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862"
Jul 08 03:49:19.425643 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.425609    2580 scope.go:117] "RemoveContainer" containerID="017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862"
Jul 08 03:49:19.426011 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:49:19.425971    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862\": container with ID starting with 017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862 not found: ID does not exist" containerID="017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862"
Jul 08 03:49:19.426081 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.426013    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862"} err="failed to get container status \"017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862\": rpc error: code = NotFound desc = could not find container \"017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862\": container with ID starting with 017e35e2d1eee2cda3faa552133f82375f898975087c8a2e7c6f9e05fd557862 not found: ID does not exist"
Jul 08 03:49:19.569913 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.569875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-credential-socket\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.569913 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.569919    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-certs\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.569938    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-data\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.569984    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-envoy\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570012    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-podinfo\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istiod-ca-cert\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb57d\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-kube-api-access-xb57d\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570200    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-socket\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570199    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570237    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:49:19.570446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-token\") pod \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\" (UID: \"2b1d4767-1cfc-4fe9-9a83-2118145f6545\") "
Jul 08 03:49:19.570842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570476    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:49:19.570842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570516    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:49:19.570842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570652    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-credential-socket\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.570842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570672    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-certs\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.570842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570687    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istiod-ca-cert\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.570842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.570700    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-workload-socket\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.571176 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.571151    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-data" (OuterVolumeSpecName: "istio-data") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:49:19.572632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.572596    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:49:19.572717 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.572646    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-kube-api-access-xb57d" (OuterVolumeSpecName: "kube-api-access-xb57d") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "kube-api-access-xb57d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:49:19.572882 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.572860    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-token" (OuterVolumeSpecName: "istio-token") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:49:19.572971 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.572934    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2b1d4767-1cfc-4fe9-9a83-2118145f6545" (UID: "2b1d4767-1cfc-4fe9-9a83-2118145f6545"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 03:49:19.671144 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.671112    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb57d\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-kube-api-access-xb57d\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.671144 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.671137    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-token\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.671144 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.671148    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-data\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.671144 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.671156    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-envoy\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.671398 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.671164    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b1d4767-1cfc-4fe9-9a83-2118145f6545-istio-podinfo\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:49:19.737408 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.737377    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"]
Jul 08 03:49:19.740966 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:19.740920    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a129cbbc-openshift-default-745c98765c-zzcwv"]
Jul 08 03:49:20.458272 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:20.458235    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" path="/var/lib/kubelet/pods/2b1d4767-1cfc-4fe9-9a83-2118145f6545/volumes"
Jul 08 03:49:21.068601 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:21.068546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:21.135767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:21.135729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:49:25.255867 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:25.255833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:25.356598 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:25.356543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:25.478908 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:25.478875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:25.579182 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:25.579151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:26.665486 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:26.665453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:26.803990 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:26.803946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:26.924297 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:26.924223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:27.223552 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:27.223465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:27.499314 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:27.499233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:27.600604 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:27.600569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:28.768067 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:28.768041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:30.043812 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:30.043772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:49:31.128119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:31.128087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:31.268187 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:31.268155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:31.372352 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:31.372320    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:31.533820 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:31.533738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:33.024691 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:33.024665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:37.204913 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:37.204815    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:37.371119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:37.371083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:37.659891 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:37.659860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:37.758521 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:37.758488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:37.859423 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:37.859390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:38.747939 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:38.747905    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:49:39.188292 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:39.188259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:44.485614 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:44.485586    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:49:44.490173 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:44.490152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:49:45.114687 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:45.114647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:45.179890 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:45.179854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:45.259881 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:45.259845    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:45.359399 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:45.359361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:45.459411 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:45.459328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:49:45.786272 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:49:45.786180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:05.128331 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:05.128298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:05.183001 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:05.182972    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:11.722283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:11.722248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:17.452478 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:17.452436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:17.553860 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:17.553817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:17.654293 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:17.654249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:33.531762 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:33.531725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:39.318747 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:39.318712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:39.381818 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:39.381784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:39.456697 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:39.456667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:39.556506 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:39.556471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:50.040968 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:50.040907    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:50.800242 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:50.800205    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:56.752488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:56.752435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:56.853140 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:56.853103    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:56.949963 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:56.949925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:57.050769 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:57.050715    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:57.803143 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:57.803085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:57.901821 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:57.901787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:58.007430 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:58.007398    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:50:59.565289 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:50:59.565259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:00.234756 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:00.234719    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:51:05.382234 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:05.382198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:05.444928 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:05.444899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:06.054490 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:06.054446    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:08.737245 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:08.737206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:51:12.061332 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:12.061291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:12.162907 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:12.162870    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:12.367907 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:12.367808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:12.585835 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:12.585803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:13.086579 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:13.086546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:15.227487 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:15.227453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:15.321804 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:15.321777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:15.396228 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:15.396194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:15.516830 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:15.516745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:15.923838 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:15.923806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:16.026760 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:16.026722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:19.709624 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:19.709590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:20.355353 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:20.355314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:20.450768 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:20.450730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:51:20.453989 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:20.453945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:20.784129 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:20.784040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:26.578374 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:26.578341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:26.678063 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:26.678030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:27.143678 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:27.143623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:27.464325 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:27.464244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:27.520114 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:27.520083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:31.540166 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:31.540126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:51:33.417346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:33.417314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:33.484567 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:33.484538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:33.670695 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:33.670599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:33.890568 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:33.890538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:34.016576 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:34.016499    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:34.109265 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:34.109232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:34.217343 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:34.217304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:39.949347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:39.949319    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:40.078810 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:40.078774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:40.193610 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:40.193575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:40.298346 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:40.298311    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:46.448509 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:46.448472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:46.562896 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:46.562864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:53.275837 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:53.275807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:53.528304 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:53.528218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:53.776910 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:53.776874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:56.355632 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:56.355598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:56.429070 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:56.429026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:56.515159 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:56.515126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:56.743235 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:56.743129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:56.928490 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:56.928462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:51:57.038329 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:51:57.038240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:00.848252 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:00.848220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:02.740228 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:02.740190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:52:06.445669 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:06.445633    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:06.546857 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:06.546814    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:10.925639 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.925602    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"]
Jul 08 03:52:10.926049 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.926032    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy"
Jul 08 03:52:10.926049 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.926043    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy"
Jul 08 03:52:10.926136 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.926126    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b1d4767-1cfc-4fe9-9a83-2118145f6545" containerName="istio-proxy"
Jul 08 03:52:10.929223 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.929206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:10.933262 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.933243    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-11ba793b\"/\"kube-root-ca.crt\""
Jul 08 03:52:10.933557 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.933539    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-11ba793b\"/\"openshift-service-ca.crt\""
Jul 08 03:52:10.934318 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.934300    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-11ba793b\"/\"default-dockercfg-tbfgq\""
Jul 08 03:52:10.934394 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.934303    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-11ba793b\"/\"e2e-lifecycle-model-11ba793b-kserve-self-signed-certs\""
Jul 08 03:52:10.945605 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.945576    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"]
Jul 08 03:52:10.961024 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.960994    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:10.984278 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.984240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-model-cache\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:10.984278 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.984276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-dshm\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:10.984493 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.984297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzb7k\" (UniqueName: \"kubernetes.io/projected/effb095c-1814-4b3d-84d2-94343fb2f872-kube-api-access-vzb7k\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:10.984493 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.984355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-home\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:10.984493 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.984395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effb095c-1814-4b3d-84d2-94343fb2f872-tls-certs\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:10.984493 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:10.984444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-tmp-dir\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085147 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-tmp-dir\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-model-cache\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-dshm\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzb7k\" (UniqueName: \"kubernetes.io/projected/effb095c-1814-4b3d-84d2-94343fb2f872-kube-api-access-vzb7k\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-home\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085347 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effb095c-1814-4b3d-84d2-94343fb2f872-tls-certs\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085619 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085553    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-model-cache\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085671 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085613    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-tmp-dir\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.085671 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.085637    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-home\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.087675 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.087652    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-dshm\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.087847 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.087827    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effb095c-1814-4b3d-84d2-94343fb2f872-tls-certs\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.092969 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.092933    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzb7k\" (UniqueName: \"kubernetes.io/projected/effb095c-1814-4b3d-84d2-94343fb2f872-kube-api-access-vzb7k\") pod \"e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") " pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.239107 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.239018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:11.372662 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:11.372638    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"]
Jul 08 03:52:11.374862 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:52:11.374834    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeffb095c_1814_4b3d_84d2_94343fb2f872.slice/crio-8933c7083bb029be73a73fd13700605789bd6f29e1575d715095a2c412b1d8af WatchSource:0}: Error finding container 8933c7083bb029be73a73fd13700605789bd6f29e1575d715095a2c412b1d8af: Status 404 returned error can't find the container with id 8933c7083bb029be73a73fd13700605789bd6f29e1575d715095a2c412b1d8af
Jul 08 03:52:12.243698 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:12.243664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd" event={"ID":"effb095c-1814-4b3d-84d2-94343fb2f872","Type":"ContainerStarted","Data":"8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224"}
Jul 08 03:52:12.243698 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:12.243701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd" event={"ID":"effb095c-1814-4b3d-84d2-94343fb2f872","Type":"ContainerStarted","Data":"8933c7083bb029be73a73fd13700605789bd6f29e1575d715095a2c412b1d8af"}
Jul 08 03:52:12.260444 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:12.260400    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd" podStartSLOduration=2.260385751 podStartE2EDuration="2.260385751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:52:12.258443658 +0000 UTC m=+1948.300628326" watchObservedRunningTime="2026-07-08 03:52:12.260385751 +0000 UTC m=+1948.302570419"
Jul 08 03:52:14.042171 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:14.042137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:52:19.232635 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:19.232594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:19.331781 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:19.331745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:21.239222 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:21.239180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:21.239621 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:21.239244    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:21.252059 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:21.252026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:21.296730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:21.296706    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:52:46.237190 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:46.237104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:52:48.581476 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:48.581441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:49.225261 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:49.225224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:49.325645 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:49.325610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:49.489307 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:49.489220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:54.645720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:54.645686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:52:59.628773 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:59.628736    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"]
Jul 08 03:52:59.724323 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:52:59.724285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:00.439693 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:00.439661    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:53:14.650584 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.650513    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd" podUID="effb095c-1814-4b3d-84d2-94343fb2f872" containerName="main" containerID="cri-o://8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224" gracePeriod=45
Jul 08 03:53:14.901653 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.901594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:53:14.961357 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-home\") pod \"effb095c-1814-4b3d-84d2-94343fb2f872\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") "
Jul 08 03:53:14.961534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzb7k\" (UniqueName: \"kubernetes.io/projected/effb095c-1814-4b3d-84d2-94343fb2f872-kube-api-access-vzb7k\") pod \"effb095c-1814-4b3d-84d2-94343fb2f872\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") "
Jul 08 03:53:14.961534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-dshm\") pod \"effb095c-1814-4b3d-84d2-94343fb2f872\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") "
Jul 08 03:53:14.961534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-tmp-dir\") pod \"effb095c-1814-4b3d-84d2-94343fb2f872\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") "
Jul 08 03:53:14.961534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961470    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-model-cache\") pod \"effb095c-1814-4b3d-84d2-94343fb2f872\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") "
Jul 08 03:53:14.961761 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effb095c-1814-4b3d-84d2-94343fb2f872-tls-certs\") pod \"effb095c-1814-4b3d-84d2-94343fb2f872\" (UID: \"effb095c-1814-4b3d-84d2-94343fb2f872\") "
Jul 08 03:53:14.961816 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961660    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-home" (OuterVolumeSpecName: "home") pod "effb095c-1814-4b3d-84d2-94343fb2f872" (UID: "effb095c-1814-4b3d-84d2-94343fb2f872"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:53:14.961816 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961719    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "effb095c-1814-4b3d-84d2-94343fb2f872" (UID: "effb095c-1814-4b3d-84d2-94343fb2f872"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:53:14.961816 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.961726    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-model-cache" (OuterVolumeSpecName: "model-cache") pod "effb095c-1814-4b3d-84d2-94343fb2f872" (UID: "effb095c-1814-4b3d-84d2-94343fb2f872"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:53:14.962150 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.962114    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-home\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:53:14.962150 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.962137    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-tmp-dir\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:53:14.962150 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.962148    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-model-cache\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:53:14.963796 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.963771    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-dshm" (OuterVolumeSpecName: "dshm") pod "effb095c-1814-4b3d-84d2-94343fb2f872" (UID: "effb095c-1814-4b3d-84d2-94343fb2f872"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:53:14.964205 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.964185    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/effb095c-1814-4b3d-84d2-94343fb2f872-kube-api-access-vzb7k" (OuterVolumeSpecName: "kube-api-access-vzb7k") pod "effb095c-1814-4b3d-84d2-94343fb2f872" (UID: "effb095c-1814-4b3d-84d2-94343fb2f872"). InnerVolumeSpecName "kube-api-access-vzb7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:53:14.964205 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:14.964190    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effb095c-1814-4b3d-84d2-94343fb2f872-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "effb095c-1814-4b3d-84d2-94343fb2f872" (UID: "effb095c-1814-4b3d-84d2-94343fb2f872"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:53:15.062821 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.062783    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effb095c-1814-4b3d-84d2-94343fb2f872-dshm\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:53:15.062821 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.062816    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effb095c-1814-4b3d-84d2-94343fb2f872-tls-certs\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:53:15.062821 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.062834    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzb7k\" (UniqueName: \"kubernetes.io/projected/effb095c-1814-4b3d-84d2-94343fb2f872-kube-api-access-vzb7k\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:53:15.537535 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.537501    2580 generic.go:356] "Generic (PLEG): container finished" podID="effb095c-1814-4b3d-84d2-94343fb2f872" containerID="8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224" exitCode=0
Jul 08 03:53:15.537720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.537540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd" event={"ID":"effb095c-1814-4b3d-84d2-94343fb2f872","Type":"ContainerDied","Data":"8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224"}
Jul 08 03:53:15.537720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.537585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd" event={"ID":"effb095c-1814-4b3d-84d2-94343fb2f872","Type":"ContainerDied","Data":"8933c7083bb029be73a73fd13700605789bd6f29e1575d715095a2c412b1d8af"}
Jul 08 03:53:15.537720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.537603    2580 scope.go:117] "RemoveContainer" containerID="8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224"
Jul 08 03:53:15.537720 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.537551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"
Jul 08 03:53:15.546822 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.546803    2580 scope.go:117] "RemoveContainer" containerID="8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224"
Jul 08 03:53:15.547103 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:53:15.547080    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224\": container with ID starting with 8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224 not found: ID does not exist" containerID="8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224"
Jul 08 03:53:15.547199 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.547111    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224"} err="failed to get container status \"8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224\": rpc error: code = NotFound desc = could not find container \"8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224\": container with ID starting with 8567ec55a0396815adc4b5b323d5c631a10bdb1a776f4cd67683741644e29224 not found: ID does not exist"
Jul 08 03:53:15.558249 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.558228    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"]
Jul 08 03:53:15.561509 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:15.561486    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-11ba793b/e2e-lifecycle-model-11ba793b-kserve-f4b475c59-xf4gd"]
Jul 08 03:53:16.459158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:16.459129    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="effb095c-1814-4b3d-84d2-94343fb2f872" path="/var/lib/kubelet/pods/effb095c-1814-4b3d-84d2-94343fb2f872/volumes"
Jul 08 03:53:36.346097 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:36.346055    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:37.542874 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:37.542839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:38.935068 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:38.935033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:53:42.786143 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:42.786110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:42.896319 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:42.896287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:43.069232 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:43.069200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:43.292246 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:43.292209    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:44.339569 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:44.339537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:44.440107 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:44.440027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:44.537801 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:44.537774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:45.272247 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:45.272179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:45.430926 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:45.430886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:45.524562 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:45.524473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:46.789586 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:46.789557    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:48.252279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:48.252240    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:53:52.471763 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:52.471724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:52.591502 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:52.591474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:53.093559 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:53.093519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:59.044123 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:59.044090    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:59.145832 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:59.145795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:59.243202 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:59.243168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:53:59.739681 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:53:59.739648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:54:09.729537 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:09.729463    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:10.056844 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:10.056817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:26.147427 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:26.147393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:30.986214 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:30.986177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:36.727940 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:36.727903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:36.789872 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:36.789842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:37.078260 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:37.078223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:37.437286 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:37.437179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:38.210666 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:38.210611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:40.081052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:40.081023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:40.168195 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:40.168163    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:40.247670 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:40.247639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:40.376308 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:40.376231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:40.660602 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:40.660514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:40.761127 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:40.761076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:44.519442 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:44.519409    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:54:44.526387 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:44.526364    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:54:44.553816 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:44.553785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:44.882367 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:44.882333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:45.180283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:45.180198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:45.381818 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:45.381785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:48.077892 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.077856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:48.161995 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.161934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:54:48.166572 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.166546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:48.261482 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.261444    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:48.407223 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.407143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:48.725470 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.725382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:48.826193 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:48.826162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:52.557034 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:52.556997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:58.332680 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.332647    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"]
Jul 08 03:54:58.333266 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.333246    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="effb095c-1814-4b3d-84d2-94343fb2f872" containerName="main"
Jul 08 03:54:58.333350 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.333269    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="effb095c-1814-4b3d-84d2-94343fb2f872" containerName="main"
Jul 08 03:54:58.333406 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.333364    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="effb095c-1814-4b3d-84d2-94343fb2f872" containerName="main"
Jul 08 03:54:58.336513 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.336493    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.339767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.339747    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-78a872\"/\"kube-root-ca.crt\""
Jul 08 03:54:58.339884 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.339797    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-78a872\"/\"default-dockercfg-rjzwf\""
Jul 08 03:54:58.339884 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.339803    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-78a872\"/\"test-model-78a872-kserve-self-signed-certs\""
Jul 08 03:54:58.340052 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.339806    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-78a872\"/\"openshift-service-ca.crt\""
Jul 08 03:54:58.348563 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.348544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"]
Jul 08 03:54:58.357175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.357156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:54:58.453719 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.453684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-model-cache\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.453868 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.453729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tmp-dir\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.453868 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.453757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbh2h\" (UniqueName: \"kubernetes.io/projected/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-kube-api-access-mbh2h\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.453868 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.453816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tls-certs\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.453868 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.453854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-dshm\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.454064 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.453880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-home\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554393 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-model-cache\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554393 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tmp-dir\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554602 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbh2h\" (UniqueName: \"kubernetes.io/projected/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-kube-api-access-mbh2h\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554602 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tls-certs\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554602 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-dshm\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554602 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-home\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554796 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554781    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-model-cache\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554856 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554828    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tmp-dir\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.554919 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.554905    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-home\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.556944 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.556926    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-dshm\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.557096 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.557080    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tls-certs\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.562583 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.562564    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbh2h\" (UniqueName: \"kubernetes.io/projected/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-kube-api-access-mbh2h\") pod \"test-model-78a872-kserve-5f5f56c659-sltc4\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") " pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.645329 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.645261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:54:58.782053 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:58.782014    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"]
Jul 08 03:54:58.784435 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:54:58.784407    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f9465a8_77e6_4c0a_9eab_77ff5ef6aac9.slice/crio-681635fe04f778857e0a7d33198b1b8a8140d708fa644d1a1d21e4e538d6279d WatchSource:0}: Error finding container 681635fe04f778857e0a7d33198b1b8a8140d708fa644d1a1d21e4e538d6279d: Status 404 returned error can't find the container with id 681635fe04f778857e0a7d33198b1b8a8140d708fa644d1a1d21e4e538d6279d
Jul 08 03:54:59.027119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:59.027032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4" event={"ID":"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9","Type":"ContainerStarted","Data":"9586e2efe69224832020ae39db387d0a8d4cecdc82c26ae36ade6ed0d3cd24a1"}
Jul 08 03:54:59.027119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:59.027070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4" event={"ID":"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9","Type":"ContainerStarted","Data":"681635fe04f778857e0a7d33198b1b8a8140d708fa644d1a1d21e4e538d6279d"}
Jul 08 03:54:59.044873 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:59.044824    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4" podStartSLOduration=1.044804201 podStartE2EDuration="1.044804201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:54:59.043471983 +0000 UTC m=+2115.085656648" watchObservedRunningTime="2026-07-08 03:54:59.044804201 +0000 UTC m=+2115.086988871"
Jul 08 03:54:59.435276 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:54:59.435242    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:55:02.319332 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:02.319298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:55:02.420241 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:02.420213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:55:08.437329 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:08.437298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:55:08.645893 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:08.645859    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:55:08.646071 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:08.645905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:55:08.658293 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:08.658265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:55:09.088091 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:09.088059    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:55:10.311447 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:10.311411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:55:10.412617 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:10.412581    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:55:30.072143 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:30.072106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:55:33.423232 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:33.423203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:55:38.441444 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:38.441361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:55:43.719332 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:55:43.719295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:22.322371 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:22.322339    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"]
Jul 08 03:56:22.342709 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:22.342678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:23.357213 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:23.357177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:23.567413 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:23.567380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:23.908335 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:23.908301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:24.022515 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:24.022480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:24.122535 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:24.122505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:24.272334 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:24.272238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:27.043843 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:27.043806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:56:28.985315 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:28.985280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:56:37.343899 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.343859    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4" podUID="2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" containerName="main" containerID="cri-o://9586e2efe69224832020ae39db387d0a8d4cecdc82c26ae36ade6ed0d3cd24a1" gracePeriod=45
Jul 08 03:56:37.507427 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.507389    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" containerID="9586e2efe69224832020ae39db387d0a8d4cecdc82c26ae36ade6ed0d3cd24a1" exitCode=0
Jul 08 03:56:37.507575 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.507465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4" event={"ID":"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9","Type":"ContainerDied","Data":"9586e2efe69224832020ae39db387d0a8d4cecdc82c26ae36ade6ed0d3cd24a1"}
Jul 08 03:56:37.593451 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.593426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:56:37.738260 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tmp-dir\") pod \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") "
Jul 08 03:56:37.738260 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738210    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tls-certs\") pod \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") "
Jul 08 03:56:37.738260 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbh2h\" (UniqueName: \"kubernetes.io/projected/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-kube-api-access-mbh2h\") pod \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") "
Jul 08 03:56:37.738534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738304    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-model-cache\") pod \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") "
Jul 08 03:56:37.738534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738326    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-home\") pod \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") "
Jul 08 03:56:37.738534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738350    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-dshm\") pod \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\" (UID: \"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9\") "
Jul 08 03:56:37.738534 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738477    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" (UID: "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:37.738730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738532    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" (UID: "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:37.738730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738573    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-home" (OuterVolumeSpecName: "home") pod "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" (UID: "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:37.738730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738655    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tmp-dir\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:56:37.738730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738675    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-model-cache\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:56:37.738730 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.738691    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-home\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:56:37.740568 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.740546    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-dshm" (OuterVolumeSpecName: "dshm") pod "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" (UID: "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:37.740672 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.740570    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-kube-api-access-mbh2h" (OuterVolumeSpecName: "kube-api-access-mbh2h") pod "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" (UID: "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9"). InnerVolumeSpecName "kube-api-access-mbh2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:56:37.740672 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.740641    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" (UID: "2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:56:37.839505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.839473    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-tls-certs\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:56:37.839505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.839500    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbh2h\" (UniqueName: \"kubernetes.io/projected/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-kube-api-access-mbh2h\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:56:37.839505 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:37.839509    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9-dshm\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:56:38.513608 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:38.513574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4" event={"ID":"2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9","Type":"ContainerDied","Data":"681635fe04f778857e0a7d33198b1b8a8140d708fa644d1a1d21e4e538d6279d"}
Jul 08 03:56:38.514010 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:38.513623    2580 scope.go:117] "RemoveContainer" containerID="9586e2efe69224832020ae39db387d0a8d4cecdc82c26ae36ade6ed0d3cd24a1"
Jul 08 03:56:38.514010 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:38.513623    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"
Jul 08 03:56:38.529087 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:38.529062    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"]
Jul 08 03:56:38.535113 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:38.535090    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-78a872/test-model-78a872-kserve-5f5f56c659-sltc4"]
Jul 08 03:56:40.458625 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:56:40.458592    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" path="/var/lib/kubelet/pods/2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9/volumes"
Jul 08 03:57:05.721168 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:05.721082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:05.987360 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:05.987277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:06.211046 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:06.211012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:06.460411 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:06.460381    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:06.873317 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:06.873284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:07.066852 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:07.066816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:07.167829 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:07.167750    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:07.319407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:07.319375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:10.232654 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:10.232618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:57:12.607111 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:12.607078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:12.670280 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:12.670251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:12.867222 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:12.867137    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:13.076575 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:13.076544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:13.417570 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:13.417537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:13.884924 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:13.884892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:14.322155 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:14.322123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:19.634074 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:19.634039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:19.738842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:19.738814    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:20.042119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.042087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:20.139603 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.139566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:20.339591 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.339517    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"]
Jul 08 03:57:20.339897 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.339885    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" containerName="main"
Jul 08 03:57:20.339943 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.339899    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" containerName="main"
Jul 08 03:57:20.340006 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.339991    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9465a8-77e6-4c0a-9eab-77ff5ef6aac9" containerName="main"
Jul 08 03:57:20.342979 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.342940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.345415 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.345390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-19508f-serving-cert\""
Jul 08 03:57:20.345583 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.345428    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2k77n\""
Jul 08 03:57:20.352835 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.352812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"]
Jul 08 03:57:20.432779 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.432741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-openshift-service-ca\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.432973 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.432804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.432973 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.432910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnntw\" (UniqueName: \"kubernetes.io/projected/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-kube-api-access-lnntw\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.438165 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.438144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:20.533846 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.533815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnntw\" (UniqueName: \"kubernetes.io/projected/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-kube-api-access-lnntw\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.534061 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.533867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-openshift-service-ca\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.534061 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.533989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.534202 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:57:20.534130    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-19508f-serving-cert: secret "maas-api-e2e-api-a-19508f-serving-cert" not found
Jul 08 03:57:20.534261 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:57:20.534213    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls podName:506ccf1e-1df0-4c9c-b50a-846663fe3f8f nodeName:}" failed. No retries permitted until 2026-07-08 03:57:21.034191038 +0000 UTC m=+2257.076375685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls") pod "maas-api-e2e-api-a-19508f-67db84d497-7mrgs" (UID: "506ccf1e-1df0-4c9c-b50a-846663fe3f8f") : secret "maas-api-e2e-api-a-19508f-serving-cert" not found
Jul 08 03:57:20.534578 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.534559    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-openshift-service-ca\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.542066 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.542045    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnntw\" (UniqueName: \"kubernetes.io/projected/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-kube-api-access-lnntw\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:20.655157 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.655057    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:20.845379 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.845348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:20.944108 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:20.944030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:21.038358 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:21.038311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:21.040918 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:21.040896    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls\") pod \"maas-api-e2e-api-a-19508f-67db84d497-7mrgs\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") " pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:21.254060 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:21.253980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:21.383571 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:21.383545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"]
Jul 08 03:57:21.385874 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:57:21.385844    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod506ccf1e_1df0_4c9c_b50a_846663fe3f8f.slice/crio-03064d1c478eb68c80969990e6e339ba568e88cee4b9036d11467ad99910bb9c WatchSource:0}: Error finding container 03064d1c478eb68c80969990e6e339ba568e88cee4b9036d11467ad99910bb9c: Status 404 returned error can't find the container with id 03064d1c478eb68c80969990e6e339ba568e88cee4b9036d11467ad99910bb9c
Jul 08 03:57:21.387769 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:21.387750    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:57:21.722484 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:21.722448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs" event={"ID":"506ccf1e-1df0-4c9c-b50a-846663fe3f8f","Type":"ContainerStarted","Data":"03064d1c478eb68c80969990e6e339ba568e88cee4b9036d11467ad99910bb9c"}
Jul 08 03:57:23.734350 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:23.734316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs" event={"ID":"506ccf1e-1df0-4c9c-b50a-846663fe3f8f","Type":"ContainerStarted","Data":"862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5"}
Jul 08 03:57:23.734738 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:23.734458    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:23.751822 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:23.751775    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs" podStartSLOduration=1.864601175 podStartE2EDuration="3.751762053s" totalImagesPullingTime="1.887160878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:57:23.75067893 +0000 UTC m=+2259.792863602" watchObservedRunningTime="2026-07-08 03:57:23.751762053 +0000 UTC m=+2259.793946722"
Jul 08 03:57:24.135264 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:24.135234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:24.658800 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:24.658764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:24.920733 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:24.920659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:25.241225 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:25.241138    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:57:25.293781 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:25.293744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:27.819829 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:27.819798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:27.920003 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:27.919940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:27.940764 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:27.940732    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"]
Jul 08 03:57:27.944768 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:27.944742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:27.947188 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:27.947166    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-42da32-serving-cert\""
Jul 08 03:57:27.953722 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:27.953698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"]
Jul 08 03:57:28.019174 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.019144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:28.102158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.102057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.102158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.102096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcf4971-3d69-4e69-873d-7d68dd2bde00-openshift-service-ca\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.102158 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.102123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bmb6\" (UniqueName: \"kubernetes.io/projected/cbcf4971-3d69-4e69-873d-7d68dd2bde00-kube-api-access-7bmb6\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.202925 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.202881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.203119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.202933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcf4971-3d69-4e69-873d-7d68dd2bde00-openshift-service-ca\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.203119 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.202994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bmb6\" (UniqueName: \"kubernetes.io/projected/cbcf4971-3d69-4e69-873d-7d68dd2bde00-kube-api-access-7bmb6\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.203119 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:57:28.203065    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-42da32-serving-cert: secret "maas-api-e2e-api-b-42da32-serving-cert" not found
Jul 08 03:57:28.203250 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:57:28.203133    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls podName:cbcf4971-3d69-4e69-873d-7d68dd2bde00 nodeName:}" failed. No retries permitted until 2026-07-08 03:57:28.70311183 +0000 UTC m=+2264.745296477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls") pod "maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" (UID: "cbcf4971-3d69-4e69-873d-7d68dd2bde00") : secret "maas-api-e2e-api-b-42da32-serving-cert" not found
Jul 08 03:57:28.203651 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.203627    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcf4971-3d69-4e69-873d-7d68dd2bde00-openshift-service-ca\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.212230 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.212208    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bmb6\" (UniqueName: \"kubernetes.io/projected/cbcf4971-3d69-4e69-873d-7d68dd2bde00-kube-api-access-7bmb6\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.493923 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.493844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:28.596551 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.596520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:28.707877 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.707835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.710380 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.710362    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls\") pod \"maas-api-e2e-api-b-42da32-cc957f6d7-skpkn\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") " pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.857413 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.857375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:28.994103 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:28.994074    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"]
Jul 08 03:57:28.996193 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:57:28.996164    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbcf4971_3d69_4e69_873d_7d68dd2bde00.slice/crio-fb911c152aee794827e625b958019bbe6fb9d135c397978829e083950e8877b0 WatchSource:0}: Error finding container fb911c152aee794827e625b958019bbe6fb9d135c397978829e083950e8877b0: Status 404 returned error can't find the container with id fb911c152aee794827e625b958019bbe6fb9d135c397978829e083950e8877b0
Jul 08 03:57:29.745195 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:29.745111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:29.764494 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:29.764461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" event={"ID":"cbcf4971-3d69-4e69-873d-7d68dd2bde00","Type":"ContainerStarted","Data":"3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e"}
Jul 08 03:57:29.764640 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:29.764512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" event={"ID":"cbcf4971-3d69-4e69-873d-7d68dd2bde00","Type":"ContainerStarted","Data":"fb911c152aee794827e625b958019bbe6fb9d135c397978829e083950e8877b0"}
Jul 08 03:57:29.764640 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:29.764595    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:29.787221 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:29.787175    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" podStartSLOduration=2.488949727 podStartE2EDuration="2.787156374s" totalImagesPullingTime="298.206647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:57:29.787134871 +0000 UTC m=+2265.829319544" watchObservedRunningTime="2026-07-08 03:57:29.787156374 +0000 UTC m=+2265.829341043"
Jul 08 03:57:32.297789 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:32.297751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:35.779983 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:35.779253    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:40.350760 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:40.350731    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:40.640791 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:40.640709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:57:40.776601 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:40.776569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:40.877643 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:40.877611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:40.981842 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:40.981757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:41.092032 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.091997    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"]
Jul 08 03:57:41.092311 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.092281    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs" podUID="506ccf1e-1df0-4c9c-b50a-846663fe3f8f" containerName="maas-api" containerID="cri-o://862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5" gracePeriod=30
Jul 08 03:57:41.117349 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.117322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:41.346751 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.346729    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:41.421279 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.421248    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-openshift-service-ca\") pod \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") "
Jul 08 03:57:41.421727 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.421291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnntw\" (UniqueName: \"kubernetes.io/projected/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-kube-api-access-lnntw\") pod \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") "
Jul 08 03:57:41.421727 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.421359    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls\") pod \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\" (UID: \"506ccf1e-1df0-4c9c-b50a-846663fe3f8f\") "
Jul 08 03:57:41.421727 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.421650    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "506ccf1e-1df0-4c9c-b50a-846663fe3f8f" (UID: "506ccf1e-1df0-4c9c-b50a-846663fe3f8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:57:41.423672 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.423649    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "506ccf1e-1df0-4c9c-b50a-846663fe3f8f" (UID: "506ccf1e-1df0-4c9c-b50a-846663fe3f8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:57:41.423767 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.423700    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-kube-api-access-lnntw" (OuterVolumeSpecName: "kube-api-access-lnntw") pod "506ccf1e-1df0-4c9c-b50a-846663fe3f8f" (UID: "506ccf1e-1df0-4c9c-b50a-846663fe3f8f"). InnerVolumeSpecName "kube-api-access-lnntw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:57:41.522446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.522369    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-openshift-service-ca\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:57:41.522446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.522395    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnntw\" (UniqueName: \"kubernetes.io/projected/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-kube-api-access-lnntw\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:57:41.522446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.522408    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506ccf1e-1df0-4c9c-b50a-846663fe3f8f-maas-api-tls\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:57:41.822687 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.822653    2580 generic.go:356] "Generic (PLEG): container finished" podID="506ccf1e-1df0-4c9c-b50a-846663fe3f8f" containerID="862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5" exitCode=0
Jul 08 03:57:41.822869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.822723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs" event={"ID":"506ccf1e-1df0-4c9c-b50a-846663fe3f8f","Type":"ContainerDied","Data":"862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5"}
Jul 08 03:57:41.822869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.822728    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"
Jul 08 03:57:41.822869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.822765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs" event={"ID":"506ccf1e-1df0-4c9c-b50a-846663fe3f8f","Type":"ContainerDied","Data":"03064d1c478eb68c80969990e6e339ba568e88cee4b9036d11467ad99910bb9c"}
Jul 08 03:57:41.822869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.822786    2580 scope.go:117] "RemoveContainer" containerID="862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5"
Jul 08 03:57:41.832309 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.832289    2580 scope.go:117] "RemoveContainer" containerID="862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5"
Jul 08 03:57:41.832612 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:57:41.832595    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5\": container with ID starting with 862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5 not found: ID does not exist" containerID="862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5"
Jul 08 03:57:41.832667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.832620    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5"} err="failed to get container status \"862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5\": rpc error: code = NotFound desc = could not find container \"862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5\": container with ID starting with 862d7c46a5f3cbaf4035643db8f28fa0236b772888727c3647063a9202e01bb5 not found: ID does not exist"
Jul 08 03:57:41.843620 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.843597    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"]
Jul 08 03:57:41.847115 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:41.847091    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-19508f-67db84d497-7mrgs"]
Jul 08 03:57:42.457965 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:42.457923    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="506ccf1e-1df0-4c9c-b50a-846663fe3f8f" path="/var/lib/kubelet/pods/506ccf1e-1df0-4c9c-b50a-846663fe3f8f/volumes"
Jul 08 03:57:46.596869 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:46.596829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:46.698058 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:46.698025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:46.860971 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:46.860865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:47.074079 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.074045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:47.227745 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.227653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:47.603758 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.603725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:47.705556 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.705524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:47.806397 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.806366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:47.942615 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.942534    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"]
Jul 08 03:57:47.942795 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.942774    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" podUID="cbcf4971-3d69-4e69-873d-7d68dd2bde00" containerName="maas-api" containerID="cri-o://3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e" gracePeriod=30
Jul 08 03:57:47.985536 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:47.985487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:48.189679 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.189651    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:48.291141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.291060    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls\") pod \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") "
Jul 08 03:57:48.291141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.291116    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcf4971-3d69-4e69-873d-7d68dd2bde00-openshift-service-ca\") pod \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") "
Jul 08 03:57:48.291365 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.291144    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bmb6\" (UniqueName: \"kubernetes.io/projected/cbcf4971-3d69-4e69-873d-7d68dd2bde00-kube-api-access-7bmb6\") pod \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\" (UID: \"cbcf4971-3d69-4e69-873d-7d68dd2bde00\") "
Jul 08 03:57:48.291542 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.291512    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbcf4971-3d69-4e69-873d-7d68dd2bde00-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbcf4971-3d69-4e69-873d-7d68dd2bde00" (UID: "cbcf4971-3d69-4e69-873d-7d68dd2bde00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:57:48.293283 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.293259    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbcf4971-3d69-4e69-873d-7d68dd2bde00" (UID: "cbcf4971-3d69-4e69-873d-7d68dd2bde00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:57:48.293446 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.293427    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbcf4971-3d69-4e69-873d-7d68dd2bde00-kube-api-access-7bmb6" (OuterVolumeSpecName: "kube-api-access-7bmb6") pod "cbcf4971-3d69-4e69-873d-7d68dd2bde00" (UID: "cbcf4971-3d69-4e69-873d-7d68dd2bde00"). InnerVolumeSpecName "kube-api-access-7bmb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:57:48.392714 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.392682    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcf4971-3d69-4e69-873d-7d68dd2bde00-openshift-service-ca\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:57:48.392714 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.392708    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bmb6\" (UniqueName: \"kubernetes.io/projected/cbcf4971-3d69-4e69-873d-7d68dd2bde00-kube-api-access-7bmb6\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:57:48.392714 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.392718    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcf4971-3d69-4e69-873d-7d68dd2bde00-maas-api-tls\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 03:57:48.858070 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.858037    2580 generic.go:356] "Generic (PLEG): container finished" podID="cbcf4971-3d69-4e69-873d-7d68dd2bde00" containerID="3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e" exitCode=0
Jul 08 03:57:48.858488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.858081    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"
Jul 08 03:57:48.858488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.858123    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" event={"ID":"cbcf4971-3d69-4e69-873d-7d68dd2bde00","Type":"ContainerDied","Data":"3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e"}
Jul 08 03:57:48.858488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.858181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn" event={"ID":"cbcf4971-3d69-4e69-873d-7d68dd2bde00","Type":"ContainerDied","Data":"fb911c152aee794827e625b958019bbe6fb9d135c397978829e083950e8877b0"}
Jul 08 03:57:48.858488 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.858209    2580 scope.go:117] "RemoveContainer" containerID="3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e"
Jul 08 03:57:48.866704 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.866685    2580 scope.go:117] "RemoveContainer" containerID="3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e"
Jul 08 03:57:48.866984 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:57:48.866964    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e\": container with ID starting with 3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e not found: ID does not exist" containerID="3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e"
Jul 08 03:57:48.867072 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.866992    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e"} err="failed to get container status \"3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e\": rpc error: code = NotFound desc = could not find container \"3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e\": container with ID starting with 3077004ba22183fd58ebd50c764fb091847a981df8c82337e12519aa29a5886e not found: ID does not exist"
Jul 08 03:57:48.873856 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.873834    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"]
Jul 08 03:57:48.880058 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:48.880037    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-42da32-cc957f6d7-skpkn"]
Jul 08 03:57:49.936914 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:49.936875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:57:50.458611 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:50.458577    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbcf4971-3d69-4e69-873d-7d68dd2bde00" path="/var/lib/kubelet/pods/cbcf4971-3d69-4e69-873d-7d68dd2bde00/volumes"
Jul 08 03:57:53.454245 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:53.454212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:53.506431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:53.506400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:53.709069 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:53.708987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:53.912644 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:53.912607    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:54.235712 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:54.235674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:54.535007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:54.534914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:54.703407 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:54.703377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:57.888148 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.888115    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:57.949513 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.949474    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"]
Jul 08 03:57:57.950117 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.950099    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbcf4971-3d69-4e69-873d-7d68dd2bde00" containerName="maas-api"
Jul 08 03:57:57.950175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.950121    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbcf4971-3d69-4e69-873d-7d68dd2bde00" containerName="maas-api"
Jul 08 03:57:57.950175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.950143    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="506ccf1e-1df0-4c9c-b50a-846663fe3f8f" containerName="maas-api"
Jul 08 03:57:57.950175 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.950152    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="506ccf1e-1df0-4c9c-b50a-846663fe3f8f" containerName="maas-api"
Jul 08 03:57:57.950284 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.950255    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="506ccf1e-1df0-4c9c-b50a-846663fe3f8f" containerName="maas-api"
Jul 08 03:57:57.950284 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.950267    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbcf4971-3d69-4e69-873d-7d68dd2bde00" containerName="maas-api"
Jul 08 03:57:57.954431 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.954406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:57.956504 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.956484    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-e356c0-serving-cert\""
Jul 08 03:57:57.956640 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.956619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2k77n\""
Jul 08 03:57:57.961685 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.961657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"]
Jul 08 03:57:57.965093 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:57.965068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:58.039209 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.039172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:58.075368 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.075336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4hdz\" (UniqueName: \"kubernetes.io/projected/155967e5-a9ff-4785-b6b0-11a458b15e93-kube-api-access-h4hdz\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.075494 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.075388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/155967e5-a9ff-4785-b6b0-11a458b15e93-maas-api-tls\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.075494 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.075479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/155967e5-a9ff-4785-b6b0-11a458b15e93-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.153145 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.153060    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:58.176838 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.176799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/155967e5-a9ff-4785-b6b0-11a458b15e93-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.176980 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.176903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4hdz\" (UniqueName: \"kubernetes.io/projected/155967e5-a9ff-4785-b6b0-11a458b15e93-kube-api-access-h4hdz\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.176980 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.176973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/155967e5-a9ff-4785-b6b0-11a458b15e93-maas-api-tls\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.177461 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.177443    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/155967e5-a9ff-4785-b6b0-11a458b15e93-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.179519 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.179491    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/155967e5-a9ff-4785-b6b0-11a458b15e93-maas-api-tls\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.195495 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.195474    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4hdz\" (UniqueName: \"kubernetes.io/projected/155967e5-a9ff-4785-b6b0-11a458b15e93-kube-api-access-h4hdz\") pod \"maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") " pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.267261 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.267232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:58.403302 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.403238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"]
Jul 08 03:57:58.405370 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:57:58.405337    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod155967e5_a9ff_4785_b6b0_11a458b15e93.slice/crio-e3cc8623efb42c5a50278718fd24936f5561c402b8139e38e8fad1b2dfcee385 WatchSource:0}: Error finding container e3cc8623efb42c5a50278718fd24936f5561c402b8139e38e8fad1b2dfcee385: Status 404 returned error can't find the container with id e3cc8623efb42c5a50278718fd24936f5561c402b8139e38e8fad1b2dfcee385
Jul 08 03:57:58.605827 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.605787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:58.707190 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.707113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:57:58.907903 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:58.907867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm" event={"ID":"155967e5-a9ff-4785-b6b0-11a458b15e93","Type":"ContainerStarted","Data":"e3cc8623efb42c5a50278718fd24936f5561c402b8139e38e8fad1b2dfcee385"}
Jul 08 03:57:59.914411 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:59.914369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm" event={"ID":"155967e5-a9ff-4785-b6b0-11a458b15e93","Type":"ContainerStarted","Data":"a3ad3b0cbeb8b930e6aa95cc324e87c608ca3d5c11fb9714c69de0e5e4ba8299"}
Jul 08 03:57:59.914832 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:59.914487    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:57:59.934566 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:57:59.934508    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm" podStartSLOduration=2.2807080969999998 podStartE2EDuration="2.934494458s" totalImagesPullingTime="653.786361ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:57:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:57:59.932661468 +0000 UTC m=+2295.974846138" watchObservedRunningTime="2026-07-08 03:57:59.934494458 +0000 UTC m=+2295.976679128"
Jul 08 03:58:02.357423 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:02.357387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:02.668209 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:02.668125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:02.960634 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:02.960551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:03.433362 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:03.433330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:58:03.525615 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:03.525585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:05.924068 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:05.924036    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 03:58:06.083818 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.083775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:06.177786 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.177705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:06.297425 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.297388    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"]
Jul 08 03:58:06.300938 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.300917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.304179 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.304157    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-b98412-serving-cert\""
Jul 08 03:58:06.316273 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.316251    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"]
Jul 08 03:58:06.383548 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.383516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:06.452661 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.452576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.452661 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.452610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9cb3a69-1953-4ba2-99c2-a87368e161ec-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.452872 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.452739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkslb\" (UniqueName: \"kubernetes.io/projected/b9cb3a69-1953-4ba2-99c2-a87368e161ec-kube-api-access-zkslb\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.554033 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.553998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkslb\" (UniqueName: \"kubernetes.io/projected/b9cb3a69-1953-4ba2-99c2-a87368e161ec-kube-api-access-zkslb\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.554200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.554047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.554200 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.554095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9cb3a69-1953-4ba2-99c2-a87368e161ec-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.554337 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:58:06.554320    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-b98412-serving-cert: secret "maas-api-e2e-shared-b-b98412-serving-cert" not found
Jul 08 03:58:06.554401 ip-10-0-129-190 kubenswrapper[2580]: E0708 03:58:06.554390    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls podName:b9cb3a69-1953-4ba2-99c2-a87368e161ec nodeName:}" failed. No retries permitted until 2026-07-08 03:58:07.0543673 +0000 UTC m=+2303.096551947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls") pod "maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" (UID: "b9cb3a69-1953-4ba2-99c2-a87368e161ec") : secret "maas-api-e2e-shared-b-b98412-serving-cert" not found
Jul 08 03:58:06.554732 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.554703    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9cb3a69-1953-4ba2-99c2-a87368e161ec-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.563319 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.563292    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkslb\" (UniqueName: \"kubernetes.io/projected/b9cb3a69-1953-4ba2-99c2-a87368e161ec-kube-api-access-zkslb\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:06.844142 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.844104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:06.943852 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:06.943819    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:07.059141 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.059099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:07.061652 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.061624    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls\") pod \"maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") " pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:07.212067 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.211980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:07.344219 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.344194    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"]
Jul 08 03:58:07.346413 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:58:07.346389    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9cb3a69_1953_4ba2_99c2_a87368e161ec.slice/crio-c867ff0144039be90095b85b1e342ab9c33162e76a255f6d85e5bd011fd4b5b4 WatchSource:0}: Error finding container c867ff0144039be90095b85b1e342ab9c33162e76a255f6d85e5bd011fd4b5b4: Status 404 returned error can't find the container with id c867ff0144039be90095b85b1e342ab9c33162e76a255f6d85e5bd011fd4b5b4
Jul 08 03:58:07.965574 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.965534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" event={"ID":"b9cb3a69-1953-4ba2-99c2-a87368e161ec","Type":"ContainerStarted","Data":"4ac03ca989398f4a6ba90d093097b30efc292bcb16c82faa5ec03d62ff6a4d4d"}
Jul 08 03:58:07.966010 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.965595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" event={"ID":"b9cb3a69-1953-4ba2-99c2-a87368e161ec","Type":"ContainerStarted","Data":"c867ff0144039be90095b85b1e342ab9c33162e76a255f6d85e5bd011fd4b5b4"}
Jul 08 03:58:07.966010 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.965764    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:07.983769 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:07.983721    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" podStartSLOduration=1.579817215 podStartE2EDuration="1.983707575s" totalImagesPullingTime="403.89036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:58:07.982582461 +0000 UTC m=+2304.024767127" watchObservedRunningTime="2026-07-08 03:58:07.983707575 +0000 UTC m=+2304.025892244"
Jul 08 03:58:10.476089 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:10.476036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:13.976593 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:13.976561    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 03:58:15.143354 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.143322    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"]
Jul 08 03:58:15.156039 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.156006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.157038 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.157008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"]
Jul 08 03:58:15.158301 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.158277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"kube-root-ca.crt\""
Jul 08 03:58:15.159007 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.158983    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"auth-test-model-e356c0-kserve-self-signed-certs\""
Jul 08 03:58:15.159121 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.159050    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"default-dockercfg-zd2gp\""
Jul 08 03:58:15.159121 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.159104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"openshift-service-ca.crt\""
Jul 08 03:58:15.173937 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.173911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:15.330475 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.330432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-home\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.330667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.330514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-model-cache\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.330667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.330536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-dshm\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.330667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.330560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tls-certs\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.330667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.330579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntrz6\" (UniqueName: \"kubernetes.io/projected/3d55441a-f0d1-42f1-99cf-45f5226c1b02-kube-api-access-ntrz6\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.330667 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.330646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tmp-dir\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.431777 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.431690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-model-cache\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.431777 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.431731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-dshm\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.431848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tls-certs\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.431884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntrz6\" (UniqueName: \"kubernetes.io/projected/3d55441a-f0d1-42f1-99cf-45f5226c1b02-kube-api-access-ntrz6\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.431906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tmp-dir\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432005 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.431973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-home\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432214 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.432170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-model-cache\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432326 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.432307    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tmp-dir\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.432374 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.432322    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-home\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.434229 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.434198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-dshm\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.434437 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.434418    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tls-certs\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.439977 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.439939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntrz6\" (UniqueName: \"kubernetes.io/projected/3d55441a-f0d1-42f1-99cf-45f5226c1b02-kube-api-access-ntrz6\") pod \"auth-test-model-e356c0-kserve-67877c7b9f-47fs5\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") " pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.466071 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.466045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:15.597737 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:15.597712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"]
Jul 08 03:58:15.599869 ip-10-0-129-190 kubenswrapper[2580]: W0708 03:58:15.599840    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d55441a_f0d1_42f1_99cf_45f5226c1b02.slice/crio-08714eafa2ca084d92296fcaaa34020b6bd40f72003681503f6673aa062548c0 WatchSource:0}: Error finding container 08714eafa2ca084d92296fcaaa34020b6bd40f72003681503f6673aa062548c0: Status 404 returned error can't find the container with id 08714eafa2ca084d92296fcaaa34020b6bd40f72003681503f6673aa062548c0
Jul 08 03:58:16.008221 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:16.008185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5" event={"ID":"3d55441a-f0d1-42f1-99cf-45f5226c1b02","Type":"ContainerStarted","Data":"050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d"}
Jul 08 03:58:16.008404 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:16.008233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5" event={"ID":"3d55441a-f0d1-42f1-99cf-45f5226c1b02","Type":"ContainerStarted","Data":"08714eafa2ca084d92296fcaaa34020b6bd40f72003681503f6673aa062548c0"}
Jul 08 03:58:16.025960 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:16.025913    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5" podStartSLOduration=1.025898655 podStartE2EDuration="1.025898655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:58:16.023727009 +0000 UTC m=+2312.065911679" watchObservedRunningTime="2026-07-08 03:58:16.025898655 +0000 UTC m=+2312.068083302"
Jul 08 03:58:18.740941 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:18.740909    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:58:19.296066 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:19.296034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:19.395710 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:19.395677    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:25.466936 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:25.466899    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:25.467399 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:25.466944    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:25.479300 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:25.479279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:26.068167 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:26.068133    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 03:58:27.618737 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:27.618705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:27.719500 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:27.719462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:38.377436 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:38.377366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:41.835305 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:41.835266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:58:43.107008 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:43.106973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:58:56.351076 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:58:56.351036    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:59:03.774577 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:59:03.774542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 03:59:13.939863 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:59:13.939818    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:59:29.339462 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:59:29.339423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 03:59:44.549928 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:59:44.549896    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:59:44.559080 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:59:44.559063    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 03:59:45.942127 ip-10-0-129-190 kubenswrapper[2580]: I0708 03:59:45.942096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:00:00.132518 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.132482    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724720-52kwv"]
Jul 08 04:00:00.136114 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.136095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:00.146859 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.146823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724720-52kwv"]
Jul 08 04:00:00.176077 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.176040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvvbb\" (UniqueName: \"kubernetes.io/projected/dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5-kube-api-access-gvvbb\") pod \"maas-api-key-cleanup-29724720-52kwv\" (UID: \"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5\") " pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:00.277456 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.277415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvvbb\" (UniqueName: \"kubernetes.io/projected/dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5-kube-api-access-gvvbb\") pod \"maas-api-key-cleanup-29724720-52kwv\" (UID: \"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5\") " pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:00.287177 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.287144    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvvbb\" (UniqueName: \"kubernetes.io/projected/dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5-kube-api-access-gvvbb\") pod \"maas-api-key-cleanup-29724720-52kwv\" (UID: \"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5\") " pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:00.447750 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.447660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:00.584995 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:00.584937    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724720-52kwv"]
Jul 08 04:00:00.587123 ip-10-0-129-190 kubenswrapper[2580]: W0708 04:00:00.587097    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4cf365_fffa_451c_ae9f_6d00d3c3c9f5.slice/crio-45d35e9e103835e61436fee757c95722f9572ae0633c9c179d659dd869371a13 WatchSource:0}: Error finding container 45d35e9e103835e61436fee757c95722f9572ae0633c9c179d659dd869371a13: Status 404 returned error can't find the container with id 45d35e9e103835e61436fee757c95722f9572ae0633c9c179d659dd869371a13
Jul 08 04:00:01.536021 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:01.535986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv" event={"ID":"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5","Type":"ContainerStarted","Data":"45d35e9e103835e61436fee757c95722f9572ae0633c9c179d659dd869371a13"}
Jul 08 04:00:04.046821 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:04.046774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:00:04.552550 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:04.552514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv" event={"ID":"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5","Type":"ContainerStarted","Data":"9e0a13c6237e075b536ccff4cc1f62d12b178bf4aee7bfdce69257aecbc95b36"}
Jul 08 04:00:04.567818 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:04.567779    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv" podStartSLOduration=0.913815317 podStartE2EDuration="4.567765976s" totalImagesPullingTime="3.653950659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:00:04.566735744 +0000 UTC m=+2420.608920413" watchObservedRunningTime="2026-07-08 04:00:04.567765976 +0000 UTC m=+2420.609950645"
Jul 08 04:00:05.558968 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:05.558923    2580 generic.go:356] "Generic (PLEG): container finished" podID="dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5" containerID="9e0a13c6237e075b536ccff4cc1f62d12b178bf4aee7bfdce69257aecbc95b36" exitCode=0
Jul 08 04:00:05.559396 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:05.559013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv" event={"ID":"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5","Type":"ContainerDied","Data":"9e0a13c6237e075b536ccff4cc1f62d12b178bf4aee7bfdce69257aecbc95b36"}
Jul 08 04:00:06.704742 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:06.704718    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:06.724330 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:06.724306    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvvbb\" (UniqueName: \"kubernetes.io/projected/dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5-kube-api-access-gvvbb\") pod \"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5\" (UID: \"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5\") "
Jul 08 04:00:06.726605 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:06.726582    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5-kube-api-access-gvvbb" (OuterVolumeSpecName: "kube-api-access-gvvbb") pod "dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5" (UID: "dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5"). InnerVolumeSpecName "kube-api-access-gvvbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:00:06.825231 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:06.825140    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvvbb\" (UniqueName: \"kubernetes.io/projected/dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5-kube-api-access-gvvbb\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:07.569928 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:07.569900    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv"
Jul 08 04:00:07.570133 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:07.569894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724720-52kwv" event={"ID":"dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5","Type":"ContainerDied","Data":"45d35e9e103835e61436fee757c95722f9572ae0633c9c179d659dd869371a13"}
Jul 08 04:00:07.570133 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:07.570031    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45d35e9e103835e61436fee757c95722f9572ae0633c9c179d659dd869371a13"
Jul 08 04:00:19.935255 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:19.935198    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:00:34.035093 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:34.035060    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"]
Jul 08 04:00:34.081563 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:34.081531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:00:35.066560 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:35.066524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:00:35.170589 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:35.170549    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:00:36.535187 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:36.535154    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:00:48.890389 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:48.890346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:00:49.055812 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.055751    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5" podUID="3d55441a-f0d1-42f1-99cf-45f5226c1b02" containerName="main" containerID="cri-o://050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d" gracePeriod=45
Jul 08 04:00:49.316272 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.316245    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 04:00:49.323722 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.323701    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-dshm\") pod \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") "
Jul 08 04:00:49.323858 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.323727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tmp-dir\") pod \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") "
Jul 08 04:00:49.323858 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.323753    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-home\") pod \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") "
Jul 08 04:00:49.323858 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.323786    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntrz6\" (UniqueName: \"kubernetes.io/projected/3d55441a-f0d1-42f1-99cf-45f5226c1b02-kube-api-access-ntrz6\") pod \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") "
Jul 08 04:00:49.323858 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.323839    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-model-cache\") pod \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") "
Jul 08 04:00:49.324108 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.323865    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tls-certs\") pod \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\" (UID: \"3d55441a-f0d1-42f1-99cf-45f5226c1b02\") "
Jul 08 04:00:49.324108 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.324028    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-home" (OuterVolumeSpecName: "home") pod "3d55441a-f0d1-42f1-99cf-45f5226c1b02" (UID: "3d55441a-f0d1-42f1-99cf-45f5226c1b02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:49.324216 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.324169    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-home\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:49.324216 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.324200    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d55441a-f0d1-42f1-99cf-45f5226c1b02" (UID: "3d55441a-f0d1-42f1-99cf-45f5226c1b02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:49.324325 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.324263    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d55441a-f0d1-42f1-99cf-45f5226c1b02" (UID: "3d55441a-f0d1-42f1-99cf-45f5226c1b02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:49.326145 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.326115    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-dshm" (OuterVolumeSpecName: "dshm") pod "3d55441a-f0d1-42f1-99cf-45f5226c1b02" (UID: "3d55441a-f0d1-42f1-99cf-45f5226c1b02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:49.326254 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.326187    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d55441a-f0d1-42f1-99cf-45f5226c1b02-kube-api-access-ntrz6" (OuterVolumeSpecName: "kube-api-access-ntrz6") pod "3d55441a-f0d1-42f1-99cf-45f5226c1b02" (UID: "3d55441a-f0d1-42f1-99cf-45f5226c1b02"). InnerVolumeSpecName "kube-api-access-ntrz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:00:49.326254 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.326221    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d55441a-f0d1-42f1-99cf-45f5226c1b02" (UID: "3d55441a-f0d1-42f1-99cf-45f5226c1b02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:00:49.425194 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.425092    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-dshm\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:49.425194 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.425123    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tmp-dir\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:49.425194 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.425133    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntrz6\" (UniqueName: \"kubernetes.io/projected/3d55441a-f0d1-42f1-99cf-45f5226c1b02-kube-api-access-ntrz6\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:49.425194 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.425150    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d55441a-f0d1-42f1-99cf-45f5226c1b02-model-cache\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:49.425194 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.425159    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d55441a-f0d1-42f1-99cf-45f5226c1b02-tls-certs\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:00:49.780996 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.780886    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d55441a-f0d1-42f1-99cf-45f5226c1b02" containerID="050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d" exitCode=0
Jul 08 04:00:49.781142 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.781009    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"
Jul 08 04:00:49.781142 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.781020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5" event={"ID":"3d55441a-f0d1-42f1-99cf-45f5226c1b02","Type":"ContainerDied","Data":"050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d"}
Jul 08 04:00:49.781142 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.781052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5" event={"ID":"3d55441a-f0d1-42f1-99cf-45f5226c1b02","Type":"ContainerDied","Data":"08714eafa2ca084d92296fcaaa34020b6bd40f72003681503f6673aa062548c0"}
Jul 08 04:00:49.781142 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.781067    2580 scope.go:117] "RemoveContainer" containerID="050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d"
Jul 08 04:00:49.793485 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.793463    2580 scope.go:117] "RemoveContainer" containerID="050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d"
Jul 08 04:00:49.793762 ip-10-0-129-190 kubenswrapper[2580]: E0708 04:00:49.793742    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d\": container with ID starting with 050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d not found: ID does not exist" containerID="050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d"
Jul 08 04:00:49.793815 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.793770    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d"} err="failed to get container status \"050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d\": rpc error: code = NotFound desc = could not find container \"050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d\": container with ID starting with 050642d7bb0dbc36f5165b811c1fcd510614c4884fa3b40bbd30d2357586259d not found: ID does not exist"
Jul 08 04:00:49.805908 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.805885    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"]
Jul 08 04:00:49.807590 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:49.807569    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/auth-test-model-e356c0-kserve-67877c7b9f-47fs5"]
Jul 08 04:00:50.458250 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:50.458219    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d55441a-f0d1-42f1-99cf-45f5226c1b02" path="/var/lib/kubelet/pods/3d55441a-f0d1-42f1-99cf-45f5226c1b02/volumes"
Jul 08 04:00:53.842344 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:00:53.842310    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:01:06.837369 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:06.837332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:01:09.241854 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.241816    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"]
Jul 08 04:01:09.242467 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.242445    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5" containerName="cleanup"
Jul 08 04:01:09.242510 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.242472    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5" containerName="cleanup"
Jul 08 04:01:09.242548 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.242523    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d55441a-f0d1-42f1-99cf-45f5226c1b02" containerName="main"
Jul 08 04:01:09.242548 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.242532    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d55441a-f0d1-42f1-99cf-45f5226c1b02" containerName="main"
Jul 08 04:01:09.242666 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.242653    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5" containerName="cleanup"
Jul 08 04:01:09.242701 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.242673    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d55441a-f0d1-42f1-99cf-45f5226c1b02" containerName="main"
Jul 08 04:01:09.246374 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.246353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.249466 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.249443    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"openshift-service-ca.crt\""
Jul 08 04:01:09.249590 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.249442    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"default-dockercfg-8kqg9\""
Jul 08 04:01:09.249590 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.249536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"kube-root-ca.crt\""
Jul 08 04:01:09.249590 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.249443    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"sub-test-model-b98412-kserve-self-signed-certs\""
Jul 08 04:01:09.253260 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.253234    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"]
Jul 08 04:01:09.272172 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.272124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:01:09.295267 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.295233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-home\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.295390 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.295274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tls-certs\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.295390 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.295300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbfgw\" (UniqueName: \"kubernetes.io/projected/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-kube-api-access-jbfgw\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.295464 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.295422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-dshm\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.295464 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.295449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tmp-dir\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.295531 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.295475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-model-cache\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396114 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-dshm\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396114 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tmp-dir\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396316 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-model-cache\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396316 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-home\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396316 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tls-certs\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396316 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbfgw\" (UniqueName: \"kubernetes.io/projected/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-kube-api-access-jbfgw\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396600 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396575    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tmp-dir\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396669 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396621    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-model-cache\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.396723 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.396678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-home\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.398448 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.398430    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-dshm\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.398858 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.398828    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tls-certs\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.406419 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.406398    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbfgw\" (UniqueName: \"kubernetes.io/projected/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-kube-api-access-jbfgw\") pod \"sub-test-model-b98412-kserve-86bdfbb9c9-jggtt\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") " pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.557522 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.557495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:09.703309 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.703280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"]
Jul 08 04:01:09.705486 ip-10-0-129-190 kubenswrapper[2580]: W0708 04:01:09.705458    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63ae631f_6560_4a5e_8ceb_6767c3b1c70a.slice/crio-d4a5ec853c04984dc05566824404ef547c8f1e60d3c3092dc4f193f94d580725 WatchSource:0}: Error finding container d4a5ec853c04984dc05566824404ef547c8f1e60d3c3092dc4f193f94d580725: Status 404 returned error can't find the container with id d4a5ec853c04984dc05566824404ef547c8f1e60d3c3092dc4f193f94d580725
Jul 08 04:01:09.886061 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.885976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" event={"ID":"63ae631f-6560-4a5e-8ceb-6767c3b1c70a","Type":"ContainerStarted","Data":"8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29"}
Jul 08 04:01:09.886061 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.886027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" event={"ID":"63ae631f-6560-4a5e-8ceb-6767c3b1c70a","Type":"ContainerStarted","Data":"d4a5ec853c04984dc05566824404ef547c8f1e60d3c3092dc4f193f94d580725"}
Jul 08 04:01:09.903634 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:09.903588    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" podStartSLOduration=0.903573664 podStartE2EDuration="903.573664ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:01:09.90200143 +0000 UTC m=+2485.944186098" watchObservedRunningTime="2026-07-08 04:01:09.903573664 +0000 UTC m=+2485.945758333"
Jul 08 04:01:12.241412 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:12.241375    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:01:19.558311 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:19.558256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:19.558311 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:19.558321    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:19.571356 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:19.571330    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:19.945930 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:19.945847    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:01:31.248109 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:31.248069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:01:48.963902 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:48.963869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:01:53.850902 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:53.850863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:01:54.827385 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:54.827350    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"]
Jul 08 04:01:54.852300 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:01:54.852271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:02.329792 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:02.329736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:02.385141 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:02.385094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:05.560704 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:05.560669    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:02:08.541884 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:08.541848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:09.848268 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:09.848214    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" podUID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" containerName="main" containerID="cri-o://8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29" gracePeriod=45
Jul 08 04:02:09.927841 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:09.927807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:09.936004 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:09.935977    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" podUID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": dial tcp 10.132.0.45:8000: connect: connection refused"
Jul 08 04:02:10.101841 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.101780    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:02:10.187123 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.187087    2580 generic.go:356] "Generic (PLEG): container finished" podID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" containerID="8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29" exitCode=0
Jul 08 04:02:10.187289 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.187125    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"
Jul 08 04:02:10.187289 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.187167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" event={"ID":"63ae631f-6560-4a5e-8ceb-6767c3b1c70a","Type":"ContainerDied","Data":"8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29"}
Jul 08 04:02:10.187289 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.187210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt" event={"ID":"63ae631f-6560-4a5e-8ceb-6767c3b1c70a","Type":"ContainerDied","Data":"d4a5ec853c04984dc05566824404ef547c8f1e60d3c3092dc4f193f94d580725"}
Jul 08 04:02:10.187289 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.187228    2580 scope.go:117] "RemoveContainer" containerID="8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29"
Jul 08 04:02:10.196084 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.196064    2580 scope.go:117] "RemoveContainer" containerID="8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29"
Jul 08 04:02:10.196345 ip-10-0-129-190 kubenswrapper[2580]: E0708 04:02:10.196327    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29\": container with ID starting with 8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29 not found: ID does not exist" containerID="8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29"
Jul 08 04:02:10.196390 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.196353    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29"} err="failed to get container status \"8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29\": rpc error: code = NotFound desc = could not find container \"8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29\": container with ID starting with 8944824edf8f6b5e9306abb3bc8a90a0f627cabc649ddc2bc8571883c2055e29 not found: ID does not exist"
Jul 08 04:02:10.266859 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.266827    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tmp-dir\") pod \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") "
Jul 08 04:02:10.267018 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.266897    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-home\") pod \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") "
Jul 08 04:02:10.267018 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tls-certs\") pod \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") "
Jul 08 04:02:10.267113 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbfgw\" (UniqueName: \"kubernetes.io/projected/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-kube-api-access-jbfgw\") pod \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") "
Jul 08 04:02:10.267113 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267043    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-model-cache\") pod \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") "
Jul 08 04:02:10.267113 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267061    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-dshm\") pod \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\" (UID: \"63ae631f-6560-4a5e-8ceb-6767c3b1c70a\") "
Jul 08 04:02:10.267249 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267099    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63ae631f-6560-4a5e-8ceb-6767c3b1c70a" (UID: "63ae631f-6560-4a5e-8ceb-6767c3b1c70a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:10.267249 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267162    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-home" (OuterVolumeSpecName: "home") pod "63ae631f-6560-4a5e-8ceb-6767c3b1c70a" (UID: "63ae631f-6560-4a5e-8ceb-6767c3b1c70a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:10.267360 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267287    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-model-cache" (OuterVolumeSpecName: "model-cache") pod "63ae631f-6560-4a5e-8ceb-6767c3b1c70a" (UID: "63ae631f-6560-4a5e-8ceb-6767c3b1c70a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:10.267420 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267388    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-model-cache\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:02:10.267420 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267404    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tmp-dir\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:02:10.267420 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.267413    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-home\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:02:10.269362 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.269336    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-dshm" (OuterVolumeSpecName: "dshm") pod "63ae631f-6560-4a5e-8ceb-6767c3b1c70a" (UID: "63ae631f-6560-4a5e-8ceb-6767c3b1c70a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:10.269466 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.269369    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63ae631f-6560-4a5e-8ceb-6767c3b1c70a" (UID: "63ae631f-6560-4a5e-8ceb-6767c3b1c70a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:02:10.269466 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.269395    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-kube-api-access-jbfgw" (OuterVolumeSpecName: "kube-api-access-jbfgw") pod "63ae631f-6560-4a5e-8ceb-6767c3b1c70a" (UID: "63ae631f-6560-4a5e-8ceb-6767c3b1c70a"). InnerVolumeSpecName "kube-api-access-jbfgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:02:10.368008 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.367922    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-tls-certs\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:02:10.368008 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.367979    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbfgw\" (UniqueName: \"kubernetes.io/projected/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-kube-api-access-jbfgw\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:02:10.368008 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.367989    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63ae631f-6560-4a5e-8ceb-6767c3b1c70a-dshm\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:02:10.504959 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.504918    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"]
Jul 08 04:02:10.509020 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:10.508996    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/sub-test-model-b98412-kserve-86bdfbb9c9-jggtt"]
Jul 08 04:02:12.458373 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:12.458335    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" path="/var/lib/kubelet/pods/63ae631f-6560-4a5e-8ceb-6767c3b1c70a/volumes"
Jul 08 04:02:19.938416 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:19.938382    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:02:20.835591 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:20.835552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:20.933427 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:20.933393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:28.533674 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.533637    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"]
Jul 08 04:02:28.534132 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.534029    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" containerName="main"
Jul 08 04:02:28.534132 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.534042    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" containerName="main"
Jul 08 04:02:28.534132 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.534119    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="63ae631f-6560-4a5e-8ceb-6767c3b1c70a" containerName="main"
Jul 08 04:02:28.537162 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.537145    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.539344 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.539317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"kube-root-ca.crt\""
Jul 08 04:02:28.539473 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.539317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"openshift-service-ca.crt\""
Jul 08 04:02:28.539473 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.539321    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"default-dockercfg-8kqg9\""
Jul 08 04:02:28.540145 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.540125    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"rate-test-model-b98412-kserve-self-signed-certs\""
Jul 08 04:02:28.545836 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.545813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"]
Jul 08 04:02:28.626597 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.626566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-home\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.626597 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.626601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-dshm\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.626868 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.626626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b903398-83c0-4007-aea9-0bf3fa87db02-tls-certs\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.626868 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.626649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hdcd\" (UniqueName: \"kubernetes.io/projected/6b903398-83c0-4007-aea9-0bf3fa87db02-kube-api-access-8hdcd\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.626868 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.626708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-model-cache\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.626868 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.626833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-tmp-dir\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.642190 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.642156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:02:28.727578 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.727535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-model-cache\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.727778 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.727627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-tmp-dir\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.727778 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.727681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-home\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.727778 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.727710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-dshm\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.727778 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.727725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b903398-83c0-4007-aea9-0bf3fa87db02-tls-certs\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.727778 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.727743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hdcd\" (UniqueName: \"kubernetes.io/projected/6b903398-83c0-4007-aea9-0bf3fa87db02-kube-api-access-8hdcd\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.728098 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.728058    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-model-cache\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.728098 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.728076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-tmp-dir\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.728191 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.728126    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-home\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.730234 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.730213    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-dshm\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.730409 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.730390    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b903398-83c0-4007-aea9-0bf3fa87db02-tls-certs\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.735375 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.735352    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hdcd\" (UniqueName: \"kubernetes.io/projected/6b903398-83c0-4007-aea9-0bf3fa87db02-kube-api-access-8hdcd\") pod \"rate-test-model-b98412-kserve-554fd9b4cf-98fzg\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") " pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.847353 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.847319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:28.983782 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:28.983756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"]
Jul 08 04:02:28.985726 ip-10-0-129-190 kubenswrapper[2580]: W0708 04:02:28.985699    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b903398_83c0_4007_aea9_0bf3fa87db02.slice/crio-32a44a41204346d3cde211c6723727ebc0f99dc0cbe13a750c9afdf90c976c05 WatchSource:0}: Error finding container 32a44a41204346d3cde211c6723727ebc0f99dc0cbe13a750c9afdf90c976c05: Status 404 returned error can't find the container with id 32a44a41204346d3cde211c6723727ebc0f99dc0cbe13a750c9afdf90c976c05
Jul 08 04:02:29.282682 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:29.282590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg" event={"ID":"6b903398-83c0-4007-aea9-0bf3fa87db02","Type":"ContainerStarted","Data":"b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1"}
Jul 08 04:02:29.282682 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:29.282632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg" event={"ID":"6b903398-83c0-4007-aea9-0bf3fa87db02","Type":"ContainerStarted","Data":"32a44a41204346d3cde211c6723727ebc0f99dc0cbe13a750c9afdf90c976c05"}
Jul 08 04:02:29.299009 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:29.298941    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg" podStartSLOduration=1.2989290869999999 podStartE2EDuration="1.298929087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:02:29.298410167 +0000 UTC m=+2565.340594837" watchObservedRunningTime="2026-07-08 04:02:29.298929087 +0000 UTC m=+2565.341113755"
Jul 08 04:02:34.745871 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:34.745827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:02:38.848226 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:38.848187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:38.848628 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:38.848481    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:38.860833 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:38.860807    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:39.343338 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:39.343311    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:02:57.045557 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:02:57.045519    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:03:17.838275 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:17.838188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:03:19.970755 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:19.970716    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:20.070488 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:20.070456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:21.844055 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:21.844013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:22.929851 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:22.929756    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"]
Jul 08 04:03:22.954768 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:22.954735    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:23.981280 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:23.981248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:24.355256 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.355216    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"]
Jul 08 04:03:24.355518 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.355490    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm" podUID="155967e5-a9ff-4785-b6b0-11a458b15e93" containerName="maas-api" containerID="cri-o://a3ad3b0cbeb8b930e6aa95cc324e87c608ca3d5c11fb9714c69de0e5e4ba8299" gracePeriod=30
Jul 08 04:03:24.379692 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.379668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:24.429770 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.429733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:24.550091 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.550061    2580 generic.go:356] "Generic (PLEG): container finished" podID="155967e5-a9ff-4785-b6b0-11a458b15e93" containerID="a3ad3b0cbeb8b930e6aa95cc324e87c608ca3d5c11fb9714c69de0e5e4ba8299" exitCode=0
Jul 08 04:03:24.550290 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.550114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm" event={"ID":"155967e5-a9ff-4785-b6b0-11a458b15e93","Type":"ContainerDied","Data":"a3ad3b0cbeb8b930e6aa95cc324e87c608ca3d5c11fb9714c69de0e5e4ba8299"}
Jul 08 04:03:24.608110 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.608053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 04:03:24.753761 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.753726    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/155967e5-a9ff-4785-b6b0-11a458b15e93-openshift-service-ca\") pod \"155967e5-a9ff-4785-b6b0-11a458b15e93\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") "
Jul 08 04:03:24.753761 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.753768    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4hdz\" (UniqueName: \"kubernetes.io/projected/155967e5-a9ff-4785-b6b0-11a458b15e93-kube-api-access-h4hdz\") pod \"155967e5-a9ff-4785-b6b0-11a458b15e93\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") "
Jul 08 04:03:24.753989 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.753802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/155967e5-a9ff-4785-b6b0-11a458b15e93-maas-api-tls\") pod \"155967e5-a9ff-4785-b6b0-11a458b15e93\" (UID: \"155967e5-a9ff-4785-b6b0-11a458b15e93\") "
Jul 08 04:03:24.754089 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.754066    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/155967e5-a9ff-4785-b6b0-11a458b15e93-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "155967e5-a9ff-4785-b6b0-11a458b15e93" (UID: "155967e5-a9ff-4785-b6b0-11a458b15e93"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 04:03:24.754247 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.754230    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/155967e5-a9ff-4785-b6b0-11a458b15e93-openshift-service-ca\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:24.756028 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.756000    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/155967e5-a9ff-4785-b6b0-11a458b15e93-kube-api-access-h4hdz" (OuterVolumeSpecName: "kube-api-access-h4hdz") pod "155967e5-a9ff-4785-b6b0-11a458b15e93" (UID: "155967e5-a9ff-4785-b6b0-11a458b15e93"). InnerVolumeSpecName "kube-api-access-h4hdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:03:24.756128 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.756092    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/155967e5-a9ff-4785-b6b0-11a458b15e93-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "155967e5-a9ff-4785-b6b0-11a458b15e93" (UID: "155967e5-a9ff-4785-b6b0-11a458b15e93"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:03:24.854990 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.854942    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4hdz\" (UniqueName: \"kubernetes.io/projected/155967e5-a9ff-4785-b6b0-11a458b15e93-kube-api-access-h4hdz\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:24.854990 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:24.854985    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/155967e5-a9ff-4785-b6b0-11a458b15e93-maas-api-tls\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:25.557214 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:25.557174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm" event={"ID":"155967e5-a9ff-4785-b6b0-11a458b15e93","Type":"ContainerDied","Data":"e3cc8623efb42c5a50278718fd24936f5561c402b8139e38e8fad1b2dfcee385"}
Jul 08 04:03:25.557214 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:25.557208    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"
Jul 08 04:03:25.557717 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:25.557249    2580 scope.go:117] "RemoveContainer" containerID="a3ad3b0cbeb8b930e6aa95cc324e87c608ca3d5c11fb9714c69de0e5e4ba8299"
Jul 08 04:03:25.578584 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:25.578548    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"]
Jul 08 04:03:25.583197 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:25.583175    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-e356c0-8554c99b45-d9prm"]
Jul 08 04:03:26.458253 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:26.458212    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="155967e5-a9ff-4785-b6b0-11a458b15e93" path="/var/lib/kubelet/pods/155967e5-a9ff-4785-b6b0-11a458b15e93/volumes"
Jul 08 04:03:28.255177 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:28.255138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:28.356062 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:28.356029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:29.451228 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:29.451180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:03:33.139069 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:33.139033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:03:37.950807 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:37.950763    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg" podUID="6b903398-83c0-4007-aea9-0bf3fa87db02" containerName="main" containerID="cri-o://b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1" gracePeriod=45
Jul 08 04:03:38.201913 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.201844    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:03:38.380406 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380370    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-dshm\") pod \"6b903398-83c0-4007-aea9-0bf3fa87db02\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") "
Jul 08 04:03:38.380595 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-tmp-dir\") pod \"6b903398-83c0-4007-aea9-0bf3fa87db02\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") "
Jul 08 04:03:38.380595 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-home\") pod \"6b903398-83c0-4007-aea9-0bf3fa87db02\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") "
Jul 08 04:03:38.380595 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380479    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hdcd\" (UniqueName: \"kubernetes.io/projected/6b903398-83c0-4007-aea9-0bf3fa87db02-kube-api-access-8hdcd\") pod \"6b903398-83c0-4007-aea9-0bf3fa87db02\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") "
Jul 08 04:03:38.380595 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b903398-83c0-4007-aea9-0bf3fa87db02-tls-certs\") pod \"6b903398-83c0-4007-aea9-0bf3fa87db02\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") "
Jul 08 04:03:38.380595 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380559    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-model-cache\") pod \"6b903398-83c0-4007-aea9-0bf3fa87db02\" (UID: \"6b903398-83c0-4007-aea9-0bf3fa87db02\") "
Jul 08 04:03:38.380864 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380782    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b903398-83c0-4007-aea9-0bf3fa87db02" (UID: "6b903398-83c0-4007-aea9-0bf3fa87db02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:38.380864 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380795    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-home" (OuterVolumeSpecName: "home") pod "6b903398-83c0-4007-aea9-0bf3fa87db02" (UID: "6b903398-83c0-4007-aea9-0bf3fa87db02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:38.380995 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.380900    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b903398-83c0-4007-aea9-0bf3fa87db02" (UID: "6b903398-83c0-4007-aea9-0bf3fa87db02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:38.382718 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.382696    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-dshm" (OuterVolumeSpecName: "dshm") pod "6b903398-83c0-4007-aea9-0bf3fa87db02" (UID: "6b903398-83c0-4007-aea9-0bf3fa87db02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:38.382807 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.382779    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b903398-83c0-4007-aea9-0bf3fa87db02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b903398-83c0-4007-aea9-0bf3fa87db02" (UID: "6b903398-83c0-4007-aea9-0bf3fa87db02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:03:38.382854 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.382811    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b903398-83c0-4007-aea9-0bf3fa87db02-kube-api-access-8hdcd" (OuterVolumeSpecName: "kube-api-access-8hdcd") pod "6b903398-83c0-4007-aea9-0bf3fa87db02" (UID: "6b903398-83c0-4007-aea9-0bf3fa87db02"). InnerVolumeSpecName "kube-api-access-8hdcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:03:38.482082 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.482057    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b903398-83c0-4007-aea9-0bf3fa87db02-tls-certs\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:38.482082 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.482078    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-model-cache\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:38.482269 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.482087    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-dshm\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:38.482269 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.482095    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-tmp-dir\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:38.482269 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.482102    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b903398-83c0-4007-aea9-0bf3fa87db02-home\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:38.482269 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.482110    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hdcd\" (UniqueName: \"kubernetes.io/projected/6b903398-83c0-4007-aea9-0bf3fa87db02-kube-api-access-8hdcd\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:03:38.622054 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.622020    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b903398-83c0-4007-aea9-0bf3fa87db02" containerID="b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1" exitCode=0
Jul 08 04:03:38.622193 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.622065    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"
Jul 08 04:03:38.622193 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.622070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg" event={"ID":"6b903398-83c0-4007-aea9-0bf3fa87db02","Type":"ContainerDied","Data":"b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1"}
Jul 08 04:03:38.622193 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.622161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg" event={"ID":"6b903398-83c0-4007-aea9-0bf3fa87db02","Type":"ContainerDied","Data":"32a44a41204346d3cde211c6723727ebc0f99dc0cbe13a750c9afdf90c976c05"}
Jul 08 04:03:38.622193 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.622177    2580 scope.go:117] "RemoveContainer" containerID="b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1"
Jul 08 04:03:38.630627 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.630609    2580 scope.go:117] "RemoveContainer" containerID="b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1"
Jul 08 04:03:38.630919 ip-10-0-129-190 kubenswrapper[2580]: E0708 04:03:38.630890    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1\": container with ID starting with b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1 not found: ID does not exist" containerID="b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1"
Jul 08 04:03:38.631135 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.630928    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1"} err="failed to get container status \"b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1\": rpc error: code = NotFound desc = could not find container \"b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1\": container with ID starting with b58d5094429ec76112c0655539be20e90344af95935c948a4dd3fadea4a736d1 not found: ID does not exist"
Jul 08 04:03:38.637966 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.637923    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"]
Jul 08 04:03:38.643768 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:38.643743    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/rate-test-model-b98412-kserve-554fd9b4cf-98fzg"]
Jul 08 04:03:40.458096 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:03:40.458062    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b903398-83c0-4007-aea9-0bf3fa87db02" path="/var/lib/kubelet/pods/6b903398-83c0-4007-aea9-0bf3fa87db02/volumes"
Jul 08 04:04:06.275719 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:06.275679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:07.352337 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.352305    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"]
Jul 08 04:04:07.352817 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.352544    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" podUID="b9cb3a69-1953-4ba2-99c2-a87368e161ec" containerName="maas-api" containerID="cri-o://4ac03ca989398f4a6ba90d093097b30efc292bcb16c82faa5ec03d62ff6a4d4d" gracePeriod=30
Jul 08 04:04:07.761318 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.761284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:07.765477 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.765454    2580 generic.go:356] "Generic (PLEG): container finished" podID="b9cb3a69-1953-4ba2-99c2-a87368e161ec" containerID="4ac03ca989398f4a6ba90d093097b30efc292bcb16c82faa5ec03d62ff6a4d4d" exitCode=0
Jul 08 04:04:07.765600 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.765501    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" event={"ID":"b9cb3a69-1953-4ba2-99c2-a87368e161ec","Type":"ContainerDied","Data":"4ac03ca989398f4a6ba90d093097b30efc292bcb16c82faa5ec03d62ff6a4d4d"}
Jul 08 04:04:07.799798 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.799778    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 04:04:07.930324 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.930241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkslb\" (UniqueName: \"kubernetes.io/projected/b9cb3a69-1953-4ba2-99c2-a87368e161ec-kube-api-access-zkslb\") pod \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") "
Jul 08 04:04:07.930324 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.930283    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9cb3a69-1953-4ba2-99c2-a87368e161ec-openshift-service-ca\") pod \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") "
Jul 08 04:04:07.930324 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.930320    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls\") pod \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\" (UID: \"b9cb3a69-1953-4ba2-99c2-a87368e161ec\") "
Jul 08 04:04:07.930713 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.930689    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9cb3a69-1953-4ba2-99c2-a87368e161ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9cb3a69-1953-4ba2-99c2-a87368e161ec" (UID: "b9cb3a69-1953-4ba2-99c2-a87368e161ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 04:04:07.932619 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.932599    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9cb3a69-1953-4ba2-99c2-a87368e161ec" (UID: "b9cb3a69-1953-4ba2-99c2-a87368e161ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:04:07.932678 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:07.932655    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9cb3a69-1953-4ba2-99c2-a87368e161ec-kube-api-access-zkslb" (OuterVolumeSpecName: "kube-api-access-zkslb") pod "b9cb3a69-1953-4ba2-99c2-a87368e161ec" (UID: "b9cb3a69-1953-4ba2-99c2-a87368e161ec"). InnerVolumeSpecName "kube-api-access-zkslb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:04:08.032113 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.032079    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkslb\" (UniqueName: \"kubernetes.io/projected/b9cb3a69-1953-4ba2-99c2-a87368e161ec-kube-api-access-zkslb\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:04:08.032113 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.032113    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9cb3a69-1953-4ba2-99c2-a87368e161ec-openshift-service-ca\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:04:08.032299 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.032127    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9cb3a69-1953-4ba2-99c2-a87368e161ec-maas-api-tls\") on node \"ip-10-0-129-190.ec2.internal\" DevicePath \"\""
Jul 08 04:04:08.771523 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.771440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"
Jul 08 04:04:08.771523 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.771443    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp" event={"ID":"b9cb3a69-1953-4ba2-99c2-a87368e161ec","Type":"ContainerDied","Data":"c867ff0144039be90095b85b1e342ab9c33162e76a255f6d85e5bd011fd4b5b4"}
Jul 08 04:04:08.772032 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.771573    2580 scope.go:117] "RemoveContainer" containerID="4ac03ca989398f4a6ba90d093097b30efc292bcb16c82faa5ec03d62ff6a4d4d"
Jul 08 04:04:08.788247 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.788223    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"]
Jul 08 04:04:08.792598 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:08.792578    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b98412-7fb6bd554b-2z5bp"]
Jul 08 04:04:09.539690 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:09.539651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wfn9v"]
Jul 08 04:04:10.457769 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:10.457732    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9cb3a69-1953-4ba2-99c2-a87368e161ec" path="/var/lib/kubelet/pods/b9cb3a69-1953-4ba2-99c2-a87368e161ec/volumes"
Jul 08 04:04:13.300942 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:13.300902    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:13.372313 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:13.372273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:13.561507 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:13.561414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:13.782232 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:13.782197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:19.146020 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:19.145983    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724720-52kwv_dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5/cleanup/0.log"
Jul 08 04:04:19.607363 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:19.607337    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx_d00b7e21-ef64-4492-8345-7ae2881cb101/copy-manifests/0.log"
Jul 08 04:04:19.614176 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:19.614152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx_d00b7e21-ef64-4492-8345-7ae2881cb101/manager/0.log"
Jul 08 04:04:21.634195 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:21.634161    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7fw95_b6efde98-36d5-425d-ac7a-d6ab4ff33b86/kuadrant-console-plugin/0.log"
Jul 08 04:04:21.978547 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:21.978465    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wfn9v_7f371001-c979-47f4-a621-521f2ecb30b9/limitador/0.log"
Jul 08 04:04:22.776023 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:22.775989    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-vb2xt_80369dcb-9893-491f-bcd8-b6a7f595e19c/kube-auth-proxy/0.log"
Jul 08 04:04:23.010147 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:23.010038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qx2xb_646410fa-87db-4499-b8df-86c468fd97a7/payload-pre-processing/0.log"
Jul 08 04:04:23.800629 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:23.800594    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-kjh9f_b48757e7-36de-4df7-a603-b3ab1354a9ca/main/0.log"
Jul 08 04:04:24.128843 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:24.128757    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vsdwf_be89fffc-feb6-4496-ac2f-b23663f7fa49/main/0.log"
Jul 08 04:04:27.686332 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:27.686298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:27.787864 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:27.787827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:28.356299 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:28.356256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:28.456488 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:28.456450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log" limit=1024
Jul 08 04:04:30.822323 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:30.822286    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-txl57_122f7785-2bbc-41e8-99fe-cbc1d4c71b7b/global-pull-secret-syncer/0.log"
Jul 08 04:04:30.880471 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:30.880447    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6zmpz_53f61516-48cb-43e6-95e1-f0b62350a522/konnectivity-agent/0.log"
Jul 08 04:04:30.980456 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:30.980422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-190.ec2.internal_bd44f3e11daee88facac900a13ae9553/haproxy/0.log"
Jul 08 04:04:36.385390 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:36.385361    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7fw95_b6efde98-36d5-425d-ac7a-d6ab4ff33b86/kuadrant-console-plugin/0.log"
Jul 08 04:04:36.690525 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:36.690497    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wfn9v_7f371001-c979-47f4-a621-521f2ecb30b9/limitador/0.log"
Jul 08 04:04:38.760480 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:38.760449    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-gvdbh_78e7b8a1-4722-4273-b358-1dc16d59d957/cluster-monitoring-operator/0.log"
Jul 08 04:04:38.865891 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:38.865862    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6b999598d-cxtrh_4e24342f-4c1b-493f-8872-c6b8d082b693/metrics-server/0.log"
Jul 08 04:04:38.892925 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:38.892898    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-ndlsp_a80a4880-e7fb-48c9-9cf4-a218d2010776/monitoring-plugin/0.log"
Jul 08 04:04:38.923895 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:38.923829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sk6p_93745864-df22-4d7c-8893-266b89638f94/node-exporter/0.log"
Jul 08 04:04:38.946093 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:38.946074    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sk6p_93745864-df22-4d7c-8893-266b89638f94/kube-rbac-proxy/0.log"
Jul 08 04:04:38.967685 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:38.967665    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sk6p_93745864-df22-4d7c-8893-266b89638f94/init-textfile/0.log"
Jul 08 04:04:39.348386 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.348348    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/prometheus/0.log"
Jul 08 04:04:39.373482 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.373446    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/config-reloader/0.log"
Jul 08 04:04:39.393382 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.393355    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/thanos-sidecar/0.log"
Jul 08 04:04:39.416856 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.416826    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/kube-rbac-proxy-web/0.log"
Jul 08 04:04:39.439690 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.439662    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/kube-rbac-proxy/0.log"
Jul 08 04:04:39.463613 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.463588    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/kube-rbac-proxy-thanos/0.log"
Jul 08 04:04:39.491355 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.491332    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_34aa8295-276e-4026-90e9-aaf5d8277e33/init-config-reloader/0.log"
Jul 08 04:04:39.520139 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.520104    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-smqbf_42e7dee9-22b3-431a-a812-6b4bf095145f/prometheus-operator/0.log"
Jul 08 04:04:39.536857 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.536834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-smqbf_42e7dee9-22b3-431a-a812-6b4bf095145f/kube-rbac-proxy/0.log"
Jul 08 04:04:39.561529 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.561498    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-5w9fb_9afe4d53-cf8d-4505-a4eb-42f7146b96bf/prometheus-operator-admission-webhook/0.log"
Jul 08 04:04:39.593483 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.593458    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b54847447-fr222_afeb9812-9020-45b3-b778-66386d98814f/telemeter-client/0.log"
Jul 08 04:04:39.614436 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.614368    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b54847447-fr222_afeb9812-9020-45b3-b778-66386d98814f/reload/0.log"
Jul 08 04:04:39.642477 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.642451    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b54847447-fr222_afeb9812-9020-45b3-b778-66386d98814f/kube-rbac-proxy/0.log"
Jul 08 04:04:39.673822 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.673795    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65756b4f8f-5wnvr_4d592b34-d08a-4281-8b9f-7aaef99cb9c7/thanos-query/0.log"
Jul 08 04:04:39.694657 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.694636    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65756b4f8f-5wnvr_4d592b34-d08a-4281-8b9f-7aaef99cb9c7/kube-rbac-proxy-web/0.log"
Jul 08 04:04:39.715173 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.715141    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65756b4f8f-5wnvr_4d592b34-d08a-4281-8b9f-7aaef99cb9c7/kube-rbac-proxy/0.log"
Jul 08 04:04:39.734940 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.734921    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65756b4f8f-5wnvr_4d592b34-d08a-4281-8b9f-7aaef99cb9c7/prom-label-proxy/0.log"
Jul 08 04:04:39.758714 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.758692    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65756b4f8f-5wnvr_4d592b34-d08a-4281-8b9f-7aaef99cb9c7/kube-rbac-proxy-rules/0.log"
Jul 08 04:04:39.780276 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:39.780257    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65756b4f8f-5wnvr_4d592b34-d08a-4281-8b9f-7aaef99cb9c7/kube-rbac-proxy-metrics/0.log"
Jul 08 04:04:40.924723 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:40.924692    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-5h75z_5e7b6863-5540-4e0c-a79b-a725921fcd4e/networking-console-plugin/0.log"
Jul 08 04:04:41.271380 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271292    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"]
Jul 08 04:04:41.271676 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271664    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9cb3a69-1953-4ba2-99c2-a87368e161ec" containerName="maas-api"
Jul 08 04:04:41.271718 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271677    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9cb3a69-1953-4ba2-99c2-a87368e161ec" containerName="maas-api"
Jul 08 04:04:41.271718 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b903398-83c0-4007-aea9-0bf3fa87db02" containerName="main"
Jul 08 04:04:41.271718 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271706    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b903398-83c0-4007-aea9-0bf3fa87db02" containerName="main"
Jul 08 04:04:41.271812 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271722    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="155967e5-a9ff-4785-b6b0-11a458b15e93" containerName="maas-api"
Jul 08 04:04:41.271812 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271728    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="155967e5-a9ff-4785-b6b0-11a458b15e93" containerName="maas-api"
Jul 08 04:04:41.271812 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271795    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b903398-83c0-4007-aea9-0bf3fa87db02" containerName="main"
Jul 08 04:04:41.271812 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271805    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="155967e5-a9ff-4785-b6b0-11a458b15e93" containerName="maas-api"
Jul 08 04:04:41.271812 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.271814    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9cb3a69-1953-4ba2-99c2-a87368e161ec" containerName="maas-api"
Jul 08 04:04:41.275119 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.275098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.277290 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.277274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hvjg8\"/\"default-dockercfg-w6npk\""
Jul 08 04:04:41.277356 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.277287    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hvjg8\"/\"kube-root-ca.crt\""
Jul 08 04:04:41.277985 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.277960    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hvjg8\"/\"openshift-service-ca.crt\""
Jul 08 04:04:41.284517 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.284485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"]
Jul 08 04:04:41.333429 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.333403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-proc\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.333588 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.333437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-podres\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.333588 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.333474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x48fs\" (UniqueName: \"kubernetes.io/projected/f06934e1-b99a-4b5b-a6ed-205742bcc279-kube-api-access-x48fs\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.333588 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.333539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-sys\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.333716 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.333620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-lib-modules\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434266 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x48fs\" (UniqueName: \"kubernetes.io/projected/f06934e1-b99a-4b5b-a6ed-205742bcc279-kube-api-access-x48fs\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434266 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-sys\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434453 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-lib-modules\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434453 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-proc\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434453 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-podres\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434453 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434443    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-sys\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434580 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434451    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-proc\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434580 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434500    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-podres\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.434580 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.434500    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f06934e1-b99a-4b5b-a6ed-205742bcc279-lib-modules\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.441888 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.441866    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x48fs\" (UniqueName: \"kubernetes.io/projected/f06934e1-b99a-4b5b-a6ed-205742bcc279-kube-api-access-x48fs\") pod \"perf-node-gather-daemonset-vzktp\" (UID: \"f06934e1-b99a-4b5b-a6ed-205742bcc279\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.585788 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.585762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.722936 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.722913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"]
Jul 08 04:04:41.725212 ip-10-0-129-190 kubenswrapper[2580]: W0708 04:04:41.725184    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf06934e1_b99a_4b5b_a6ed_205742bcc279.slice/crio-4b8efbe66423e594881fd11854c70c8a8cc45b288621ecaf64a5d87f724e5ba6 WatchSource:0}: Error finding container 4b8efbe66423e594881fd11854c70c8a8cc45b288621ecaf64a5d87f724e5ba6: Status 404 returned error can't find the container with id 4b8efbe66423e594881fd11854c70c8a8cc45b288621ecaf64a5d87f724e5ba6
Jul 08 04:04:41.933047 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.933014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp" event={"ID":"f06934e1-b99a-4b5b-a6ed-205742bcc279","Type":"ContainerStarted","Data":"d8ac0004b9a50a433d4f641fa760bcd23f6673ec4b42d7928bbbe5ee72023fd3"}
Jul 08 04:04:41.933430 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.933060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp" event={"ID":"f06934e1-b99a-4b5b-a6ed-205742bcc279","Type":"ContainerStarted","Data":"4b8efbe66423e594881fd11854c70c8a8cc45b288621ecaf64a5d87f724e5ba6"}
Jul 08 04:04:41.933430 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.933196    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:41.949673 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:41.949624    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp" podStartSLOduration=0.949610137 podStartE2EDuration="949.610137ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:04:41.948692376 +0000 UTC m=+2697.990877044" watchObservedRunningTime="2026-07-08 04:04:41.949610137 +0000 UTC m=+2697.991794805"
Jul 08 04:04:42.120448 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:42.120358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-jd2qv_0dd8ee21-1e1a-44f4-b223-611fa1759e2d/download-server/0.log"
Jul 08 04:04:42.738350 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:42.738316    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-mqq6g_88fab518-b439-4c75-85bf-52139e1bd0f5/volume-data-source-validator/0.log"
Jul 08 04:04:43.853759 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:43.853730    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8rwb_a3458bec-0dc7-42ff-b199-84cbafc31675/dns/0.log"
Jul 08 04:04:43.875636 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:43.875612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8rwb_a3458bec-0dc7-42ff-b199-84cbafc31675/kube-rbac-proxy/0.log"
Jul 08 04:04:43.923131 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:43.923109    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sz6hp_5796db27-474f-4e9e-9b0a-a3a219c600b4/dns-node-resolver/0.log"
Jul 08 04:04:44.543035 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:44.543004    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwssj_1f3445b3-8eb2-4137-8a58-f1637be7d7e4/node-ca/0.log"
Jul 08 04:04:44.580640 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:44.580612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 04:04:44.590677 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:44.590656    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrq66_ed399293-2342-4edf-bf8b-2b333a686a3e/ovn-acl-logging/0.log"
Jul 08 04:04:45.640468 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:45.640438    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-vb2xt_80369dcb-9893-491f-bcd8-b6a7f595e19c/kube-auth-proxy/0.log"
Jul 08 04:04:45.740497 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:45.740457    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qx2xb_646410fa-87db-4499-b8df-86c468fd97a7/payload-pre-processing/0.log"
Jul 08 04:04:46.521817 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:46.521789    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4vvts_c1d4333f-55cc-414a-a948-48d70aef566e/serve-healthcheck-canary/0.log"
Jul 08 04:04:47.055648 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:47.055623    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8p5ht_600a1819-ff8e-4dc3-8f50-28e4b6e0b386/kube-rbac-proxy/0.log"
Jul 08 04:04:47.077065 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:47.077038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8p5ht_600a1819-ff8e-4dc3-8f50-28e4b6e0b386/exporter/0.log"
Jul 08 04:04:47.101032 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:47.101008    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8p5ht_600a1819-ff8e-4dc3-8f50-28e4b6e0b386/extractor/0.log"
Jul 08 04:04:47.948193 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:47.948166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-vzktp"
Jul 08 04:04:49.677011 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:49.676979    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724720-52kwv_dd4cf365-fffa-451c-ae9f-6d00d3c3c9f5/cleanup/0.log"
Jul 08 04:04:49.997825 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:49.997730    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx_d00b7e21-ef64-4492-8345-7ae2881cb101/manager/0.log"
Jul 08 04:04:50.024867 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:50.024843    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-6qpvx_d00b7e21-ef64-4492-8345-7ae2881cb101/copy-manifests/0.log"
Jul 08 04:04:51.773655 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:51.773611    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d464d4856-sddcv_f00a7d3c-ac59-4871-b4d8-7c200747c8ce/manager/0.log"
Jul 08 04:04:52.302729 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:52.302704    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-vb2xt_80369dcb-9893-491f-bcd8-b6a7f595e19c/kube-auth-proxy/0.log"
Jul 08 04:04:52.384599 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:52.384575    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qx2xb_646410fa-87db-4499-b8df-86c468fd97a7/payload-pre-processing/0.log"
Jul 08 04:04:53.854481 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:53.854450    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-vb2xt_80369dcb-9893-491f-bcd8-b6a7f595e19c/kube-auth-proxy/0.log"
Jul 08 04:04:53.934059 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:53.934027    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qx2xb_646410fa-87db-4499-b8df-86c468fd97a7/payload-pre-processing/0.log"
Jul 08 04:04:56.814814 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:56.814786    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-cgmwc_99ff6c39-1c63-4e9d-99df-3366f4979cdd/migrator/0.log"
Jul 08 04:04:56.834363 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:56.834328    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-cgmwc_99ff6c39-1c63-4e9d-99df-3366f4979cdd/graceful-termination/0.log"
Jul 08 04:04:57.169193 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:57.169117    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-5lrfp_68111e66-8ce8-4eec-b76b-363147c018eb/kube-storage-version-migrator-operator/1.log"
Jul 08 04:04:57.170698 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:57.170677    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-5lrfp_68111e66-8ce8-4eec-b76b-363147c018eb/kube-storage-version-migrator-operator/0.log"
Jul 08 04:04:58.327018 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.326985    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/kube-multus-additional-cni-plugins/0.log"
Jul 08 04:04:58.348565 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.348539    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/egress-router-binary-copy/0.log"
Jul 08 04:04:58.370414 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.370396    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/cni-plugins/0.log"
Jul 08 04:04:58.393320 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.393293    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/bond-cni-plugin/0.log"
Jul 08 04:04:58.413580 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.413560    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/routeoverride-cni/0.log"
Jul 08 04:04:58.434623 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.434602    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/whereabouts-cni-bincopy/0.log"
Jul 08 04:04:58.463532 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.457536    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5ppb_b2f91405-83ae-4cc0-a508-d93ff9d99c56/whereabouts-cni/0.log"
Jul 08 04:04:58.835378 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.835346    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q5l55_be3ebbcc-0f5c-4d38-bde8-008358c5b341/kube-multus/0.log"
Jul 08 04:04:58.859667 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.859639    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8cxvd_b289c98c-c888-4672-9b09-03e12ae66c52/network-metrics-daemon/0.log"
Jul 08 04:04:58.878491 ip-10-0-129-190 kubenswrapper[2580]: I0708 04:04:58.878466    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8cxvd_b289c98c-c888-4672-9b09-03e12ae66c52/kube-rbac-proxy/0.log"