Jul 14 11:06:35.321888 ip-10-0-137-65 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 11:06:35.643115 ip-10-0-137-65 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 11:06:35.643115 ip-10-0-137-65 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 11:06:35.643115 ip-10-0-137-65 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 11:06:35.643115 ip-10-0-137-65 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 11:06:35.643115 ip-10-0-137-65 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 11:06:35.644580 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.644455    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 11:06:35.649115 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649091    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:06:35.649115 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649110    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:06:35.649115 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649115    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649124    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649130    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649135    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649139    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649143    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649146    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649149    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649153    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649157    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649161    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649165    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649169    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649173    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649176    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649180    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649184    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649188    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649192    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:06:35.649324 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649196    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649199    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649203    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649207    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649211    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649216    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649219    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649226    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649231    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649236    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649240    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649258    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649263    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649267    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649271    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649276    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649281    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649285    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649289    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649293    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:06:35.650143 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649298    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649303    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649307    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649311    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649315    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649319    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649323    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649327    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649331    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649335    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649339    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649343    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649348    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649353    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649357    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649362    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649366    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649370    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649374    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649378    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:06:35.651004 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649383    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649387    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649391    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649395    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649400    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649404    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649408    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649414    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649418    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649422    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649426    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649431    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649435    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649440    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649444    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649448    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649452    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649457    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649460    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649465    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:06:35.651848 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649469    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649473    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649479    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649483    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649487    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649491    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649495    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649499    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649504    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649510    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649516    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649521    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649525    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649530    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649534    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649538    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649543    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.649547    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650708    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650719    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:06:35.652708 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650724    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650728    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650733    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650738    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650742    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650746    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650751    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650755    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650759    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650763    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650768    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650772    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650776    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650780    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650785    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650790    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650794    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650798    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650803    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650807    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:06:35.653593 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650811    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650814    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650819    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650824    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650828    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650832    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650836    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650841    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650845    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650849    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650853    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650857    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650861    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650866    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650870    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650874    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650878    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650882    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650886    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:06:35.654208 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650891    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650894    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650898    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650903    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650907    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650911    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650915    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650919    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650923    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650928    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650933    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650938    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650941    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650945    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650949    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650953    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650959    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650964    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650969    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650974    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:06:35.654771 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650978    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650982    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650986    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650990    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650995    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.650999    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651003    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651007    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651011    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651015    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651019    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651023    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651027    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651035    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651043    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651049    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651054    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651059    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651063    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:06:35.655492 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651067    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651071    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651076    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651081    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651085    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651089    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651093    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651097    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651101    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651106    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651110    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651114    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651118    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651122    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651128    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651134    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651138    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651142    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651147    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651732    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:06:35.655986 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651740    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651744    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651748    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651753    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651757    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651762    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651766    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651770    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651774    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651778    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651783    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651787    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651791    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651795    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651800    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651805    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651809    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651813    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651817    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651821    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:06:35.656509 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651825    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651830    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651834    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651838    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651842    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651847    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651851    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651855    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651859    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651863    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651867    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651871    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651875    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651879    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651883    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651888    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651892    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651896    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651900    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:06:35.657052 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651904    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651908    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651912    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651916    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651920    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651924    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651929    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651933    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651937    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651942    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651946    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651950    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651959    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651965    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651970    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651975    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651979    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.651983    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652017    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652023    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:06:35.657529 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652028    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652033    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652037    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652041    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652045    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652057    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652061    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652066    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652070    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652075    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652079    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652083    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652087    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652091    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652095    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652099    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652104    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652108    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652113    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652117    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:06:35.658023 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652125    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652130    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652144    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652149    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652153    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652157    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652162    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652168    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652173    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652177    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652182    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652186    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652190    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652195    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652199    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652203    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652207    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652211    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.652216    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652318    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652329    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 11:06:35.658487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652339    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652348    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652354    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652359    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652366    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652372    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652378    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652383    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652388    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652393    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652399    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652403    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652408    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652413    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652419    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652424    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652431    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652436    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652441    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652446    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652451    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652457    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652462    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 11:06:35.659003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652467    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652472    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652477    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652481    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652486    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652491    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652495    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652502    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652506    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652511    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652516    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652521    2563 flags.go:64] FLAG: --enable-server="true"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652525    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652532    2563 flags.go:64] FLAG: --event-burst="100"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652537    2563 flags.go:64] FLAG: --event-qps="50"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652541    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652546    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652551    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652559    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652564    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652571    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652576    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652581    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652585    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652591    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 11:06:35.659617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652595    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652600    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652605    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652609    2563 flags.go:64] FLAG: --feature-gates=""
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652615    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652620    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652625    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652629    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652634    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652639    2563 flags.go:64] FLAG: --help="false"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652643    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652649    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652670    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652675    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652681    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652687    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652691    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652696    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652701    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652705    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652710    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652715    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652719    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652724    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 11:06:35.660303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652728    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652734    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652739    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652744    2563 flags.go:64] FLAG: --lock-file=""
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652748    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652753    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652758    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652766    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652772    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652776    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652781    2563 flags.go:64] FLAG: --logging-format="text"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652785    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652790    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652795    2563 flags.go:64] FLAG: --manifest-url=""
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652799    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652806    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652810    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652816    2563 flags.go:64] FLAG: --max-pods="110"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652821    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652825    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652830    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652835    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652839    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652844    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652849    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 11:06:35.660900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652859    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652864    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652869    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652873    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652878    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652885    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652890    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652894    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652901    2563 flags.go:64] FLAG: --port="10250"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652906    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652911    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b8a22dd64ca84f42"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652916    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652921    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652926    2563 flags.go:64] FLAG: --register-node="true"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652930    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652936    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652942    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652947    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652952    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652957    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652962    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652967    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652971    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652976    2563 flags.go:64] FLAG: --runonce="false"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652980    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 11:06:35.661489 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652985    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652991    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.652995    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653000    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653004    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653009    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653014    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653019    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653024    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653028    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653033    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653038    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653043    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653047    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653056    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653062    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653067    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653074    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653079    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653083    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653088    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653092    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653097    2563 flags.go:64] FLAG: --v="2"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653110    2563 flags.go:64] FLAG: --version="false"
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653117    2563 flags.go:64] FLAG: --vmodule=""
Jul 14 11:06:35.662105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653123    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.653128    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653302    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653309    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653314    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653319    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653323    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653328    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653333    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653337    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653342    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653346    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653351    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653355    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653359    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653363    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653370    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653376    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653380    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653383    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:06:35.662814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653387    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653392    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653398    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653402    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653407    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653411    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653415    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653419    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653423    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653427    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653431    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653435    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653441    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653445    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653449    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653453    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653457    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653461    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653466    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:06:35.663301 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653470    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653476    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653482    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653487    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653492    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653497    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653501    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653505    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653510    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653514    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653518    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653522    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653526    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653531    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653535    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653541    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653545    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653551    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653555    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:06:35.663821 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653559    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653564    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653568    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653572    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653576    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653580    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653585    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653589    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653594    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653597    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653601    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653605    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653609    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653613    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653617    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653621    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653625    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653630    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653634    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653638    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:06:35.664294 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653643    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653647    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653669    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653674    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653680    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653685    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653690    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653694    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653701    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653705    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653711    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653716    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653721    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653725    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653729    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653734    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653739    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653743    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653747    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:06:35.664795 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653752    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653758    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653762    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.653766    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.654319    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.661107    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.661126    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.661770    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.661781    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661855    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661860    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661863    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661866    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661870    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:06:35.665257 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661874    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661877    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661880    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661882    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661885    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661888    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661890    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661893    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661896    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661898    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661901    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661903    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661906    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661908    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661911    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661914    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661916    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661919    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661921    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:06:35.665597 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661924    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661926    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661929    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661931    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661934    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661943    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661947    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661949    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661952    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661954    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661957    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661959    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661962    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661964    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661967    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661969    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661972    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661974    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661977    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661979    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:06:35.666066 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661982    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661985    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661988    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661990    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661992    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661995    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.661997    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662000    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662002    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662006    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662009    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662012    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662016    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662018    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662021    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662023    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662025    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662029    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662040    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662043    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:06:35.666548 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662046    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662049    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662052    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662054    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662057    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662059    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662062    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662065    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662067    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662070    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662072    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662074    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662077    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662079    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662083    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662086    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662089    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662092    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662095    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:06:35.667095 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662098    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662100    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662103    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662105    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662108    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662111    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662114    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662116    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662119    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662122    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662126    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662128    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662137    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662140    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662142    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:06:35.667535 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662145    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.662150    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662292    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662296    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662299    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662302    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662305    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662309    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662312    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662316    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662318    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662321    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662323    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662326    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662329    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:06:35.667911 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662331    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662334    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662336    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662339    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662342    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662344    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662347    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662349    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662352    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662354    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662357    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662359    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662362    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662364    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662367    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662376    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662379    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662381    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662383    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:06:35.668284 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662386    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662389    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662391    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662393    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662396    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662398    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662401    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662403    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662406    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662408    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662410    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662413    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662415    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662418    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662420    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662422    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662425    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662428    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662430    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662433    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:06:35.668749 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662435    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662438    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662441    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662445    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662448    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662450    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662453    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662456    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662458    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662467    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662469    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662473    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662476    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662479    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662482    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662485    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662487    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662490    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662492    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:06:35.669212 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662495    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662497    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662500    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662502    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662505    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662508    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662510    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662512    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662515    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662518    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662520    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662523    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662525    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662528    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662530    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662533    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662535    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662538    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662541    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662543    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:06:35.669685 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662546    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662548    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662551    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662559    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662561    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662564    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662566    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:35.662569    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.662573    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.662709    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.664832    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.666139    2563 server.go:1019] "Starting client certificate rotation"
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.666240    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 11:06:35.670149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.666273    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 11:06:35.670463 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.668410    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 11:06:35.672638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.672616    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 11:06:35.681274 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.681258    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 14 11:06:35.683157 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.683139    2563 log.go:25] "Validated CRI v1 image API"
Jul 14 11:06:35.684031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.684017    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 11:06:35.687379 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.687358    2563 fs.go:135] Filesystem UUIDs: map[58bba5d5-1cb8-48d6-990c-c08b9ee9e4f1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d68ecc9c-1869-453f-8bf3-c430fb711195:/dev/nvme0n1p4]
Jul 14 11:06:35.687452 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.687378    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 11:06:35.692363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.692246    2563 manager.go:216] Machine: {Timestamp:2026-07-14 11:06:35.691105548 +0000 UTC m=+0.273644564 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3142285 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0ca9008446d54d4da218acb9e35c SystemUUID:ec2d0ca9-0084-46d5-4d4d-a218acb9e35c BootID:ec749529-d83f-4b13-be5a-2d743a02e58c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:b0:f7:6e:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:b0:f7:6e:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:bd:31:61:f4:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 11:06:35.692363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.692352    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 11:06:35.692520 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.692438    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 11:06:35.695505 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.695479    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 11:06:35.695680 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.695507    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-65.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 11:06:35.696018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.696003    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 11:06:35.696418 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.696406    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 11:06:35.696477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.696424    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 11:06:35.696477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.696435    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 11:06:35.697015 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.697003    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 11:06:35.698328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.698317    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 11:06:35.702151 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702138    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 11:06:35.702280 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702269    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 11:06:35.702346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702304    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 11:06:35.702346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702317    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 11:06:35.702346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702336    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 11:06:35.702477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702356    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 14 11:06:35.702477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702377    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 11:06:35.702842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702823    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 11:06:35.702906 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.702846    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 11:06:35.705247 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.705227    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 11:06:35.707581 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707563    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 11:06:35.707647 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707615    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 11:06:35.707825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707813    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 11:06:35.707870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707830    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 11:06:35.707870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707836    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 11:06:35.707870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707842    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 11:06:35.707870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707850    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 11:06:35.707870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.707862    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 11:06:35.708370 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.708360    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 11:06:35.708409 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.708372    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 11:06:35.708409 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.708380    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 11:06:35.708569 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.708558    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 11:06:35.708602 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.708577    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 11:06:35.708602 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.708589    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 11:06:35.709370 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.709359    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 11:06:35.709410 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.709371    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 11:06:35.712403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.712390    2563 server.go:1268] "Started kubelet"
Jul 14 11:06:35.712534 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.712475    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 11:06:35.712623 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.712551    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 11:06:35.712623 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.712527    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 11:06:35.713131 ip-10-0-137-65 systemd[1]: Started Kubernetes Kubelet.
Jul 14 11:06:35.714641 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.714626    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 11:06:35.715694 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.715673    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-65.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 11:06:35.715820 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.715745    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 11:06:35.715884 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.715826    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-65.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 11:06:35.716593 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.716578    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 11:06:35.717326 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.717308    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 11:06:35.719931 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.719225    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-65.ec2.internal.18c2234846125f81  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-65.ec2.internal,UID:ip-10-0-137-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-65.ec2.internal,},FirstTimestamp:2026-07-14 11:06:35.712364417 +0000 UTC m=+0.294903412,LastTimestamp:2026-07-14 11:06:35.712364417 +0000 UTC m=+0.294903412,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-65.ec2.internal,}"
Jul 14 11:06:35.721893 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.721876    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 11:06:35.721893 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.721888    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 11:06:35.722016 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.722002    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:35.722540 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722510    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 11:06:35.722637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722543    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 11:06:35.722637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722547    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 11:06:35.722637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722625    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 11:06:35.722637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722633    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 11:06:35.722947 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722932    2563 factory.go:55] Registering systemd factory
Jul 14 11:06:35.723004 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.722969    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 14 11:06:35.723203 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.723188    2563 factory.go:153] Registering CRI-O factory
Jul 14 11:06:35.723203 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.723201    2563 factory.go:223] Registration of the crio container factory successfully
Jul 14 11:06:35.723368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.723244    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 11:06:35.723368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.723263    2563 factory.go:103] Registering Raw factory
Jul 14 11:06:35.723368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.723275    2563 manager.go:1196] Started watching for new ooms in manager
Jul 14 11:06:35.723605 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.723580    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-65.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 11:06:35.723695 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.723608    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 11:06:35.724403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.724385    2563 manager.go:349] Starting recovery of all containers
Jul 14 11:06:35.734984 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.734967    2563 manager.go:354] Recovery completed
Jul 14 11:06:35.740496 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.740391    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.742381 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742353    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.742439 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742396    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.742439 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742407    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.742515 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742368    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2pk2h"
Jul 14 11:06:35.742978 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742962    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 11:06:35.742978 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742975    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 11:06:35.743071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742990    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 11:06:35.743108 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.742976    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.743740 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.743725    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.743822 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.743755    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.743822 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.743768    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.744612 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.744553    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-65.ec2.internal.18c2234847dc7a52  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-65.ec2.internal,UID:ip-10-0-137-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-65.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-65.ec2.internal,},FirstTimestamp:2026-07-14 11:06:35.74238677 +0000 UTC m=+0.324925771,LastTimestamp:2026-07-14 11:06:35.74238677 +0000 UTC m=+0.324925771,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-65.ec2.internal,}"
Jul 14 11:06:35.745148 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.745137    2563 policy_none.go:49] "None policy: Start"
Jul 14 11:06:35.745201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.745153    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 11:06:35.745201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.745163    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 11:06:35.747172 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.747162    2563 policy_none.go:47] "Start"
Jul 14 11:06:35.750527 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.750417    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2pk2h"
Jul 14 11:06:35.756444 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.755640    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-65.ec2.internal.18c2234847dcb6b5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-65.ec2.internal,UID:ip-10-0-137-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-65.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-65.ec2.internal,},FirstTimestamp:2026-07-14 11:06:35.742402229 +0000 UTC m=+0.324941223,LastTimestamp:2026-07-14 11:06:35.742402229 +0000 UTC m=+0.324941223,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-65.ec2.internal,}"
Jul 14 11:06:35.780245 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780230    2563 manager.go:337] "Starting Device Plugin manager"
Jul 14 11:06:35.780319 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.780269    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 11:06:35.780319 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780280    2563 server.go:87] "Starting device plugin registration server"
Jul 14 11:06:35.780472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780464    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 11:06:35.780519 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780475    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 11:06:35.780600 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780584    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 11:06:35.781053 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780676    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 11:06:35.781053 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.780690    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 11:06:35.781184 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.781128    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 11:06:35.781184 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.781174    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:35.817985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.817965    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 11:06:35.825499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.819081    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 11:06:35.825499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.819112    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 11:06:35.825499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.819129    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 11:06:35.825499 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.819157    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 11:06:35.825499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.821121    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:06:35.880771 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.880752    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.883413 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.883399    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.883487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.883427    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.883487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.883438    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.883487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.883459    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.891437 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.891423    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.891491 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.891443    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-65.ec2.internal\": node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:35.919988 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.919939    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"]
Jul 14 11:06:35.920073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.919998    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.920165 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.920150    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:35.923694 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.923676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/979d7968c8497a3b8cb9fe00c5e6ba4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal\" (UID: \"979d7968c8497a3b8cb9fe00c5e6ba4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.923694 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.923700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/979d7968c8497a3b8cb9fe00c5e6ba4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal\" (UID: \"979d7968c8497a3b8cb9fe00c5e6ba4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.923845 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.923728    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.923845 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.923752    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.923845 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.923764    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.926141 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.926129    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.926238 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.926227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.926277 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.926251    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.928881 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.928859    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.928881 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.928884    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.928991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.928896    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.928991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.928938    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.928991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.928957    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.928991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.928966    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.931261 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.931247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.931309 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.931272    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:35.931929 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.931914    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:35.932000 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.931939    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:35.932000 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:35.931951    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:35.954110 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.954092    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-65.ec2.internal\" not found" node="ip-10-0-137-65.ec2.internal"
Jul 14 11:06:35.958263 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:35.958248    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-65.ec2.internal\" not found" node="ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.021002 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.020980    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.024510 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.024493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/979d7968c8497a3b8cb9fe00c5e6ba4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal\" (UID: \"979d7968c8497a3b8cb9fe00c5e6ba4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.024559 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.024518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/979d7968c8497a3b8cb9fe00c5e6ba4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal\" (UID: \"979d7968c8497a3b8cb9fe00c5e6ba4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.024559 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.024539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ace48c1090b8dbb21683e27f4527c23e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-65.ec2.internal\" (UID: \"ace48c1090b8dbb21683e27f4527c23e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.024634 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.024572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/979d7968c8497a3b8cb9fe00c5e6ba4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal\" (UID: \"979d7968c8497a3b8cb9fe00c5e6ba4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.024634 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.024590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/979d7968c8497a3b8cb9fe00c5e6ba4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal\" (UID: \"979d7968c8497a3b8cb9fe00c5e6ba4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.128082 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.128048    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.128174 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.128081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ace48c1090b8dbb21683e27f4527c23e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-65.ec2.internal\" (UID: \"ace48c1090b8dbb21683e27f4527c23e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.128174 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.128133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ace48c1090b8dbb21683e27f4527c23e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-65.ec2.internal\" (UID: \"ace48c1090b8dbb21683e27f4527c23e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.229233 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.229175    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.257501 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.257478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.261410 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.261389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:36.330354 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.330328    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.430536 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.430506    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.531641 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.531579    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.632697 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.632672    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.667456 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.667435    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 11:06:36.667915 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.667576    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 11:06:36.733646 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.733619    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.754807 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.754772    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 11:01:36 +0000 UTC" deadline="2028-03-18 15:42:26.672893928 +0000 UTC"
Jul 14 11:06:36.754807 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.754800    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14716h35m49.91809757s"
Jul 14 11:06:36.783005 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.782953    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:06:36.833800 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.833779    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.873585 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:36.873556    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace48c1090b8dbb21683e27f4527c23e.slice/crio-80c4a9911c262515e3eab9a151f7dca0073af8c4ca0a829792dc3f0d1b9ec4b7 WatchSource:0}: Error finding container 80c4a9911c262515e3eab9a151f7dca0073af8c4ca0a829792dc3f0d1b9ec4b7: Status 404 returned error can't find the container with id 80c4a9911c262515e3eab9a151f7dca0073af8c4ca0a829792dc3f0d1b9ec4b7
Jul 14 11:06:36.874000 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:36.873983    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979d7968c8497a3b8cb9fe00c5e6ba4c.slice/crio-200de1858ba8f34b6623c02e6914ff8f1079b0cd9652fa03cb14b34637f69fe2 WatchSource:0}: Error finding container 200de1858ba8f34b6623c02e6914ff8f1079b0cd9652fa03cb14b34637f69fe2: Status 404 returned error can't find the container with id 200de1858ba8f34b6623c02e6914ff8f1079b0cd9652fa03cb14b34637f69fe2
Jul 14 11:06:36.877620 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.877598    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:06:36.923735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.923713    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:06:36.924934 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.924915    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:06:36.925017 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.924947    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:06:36.925017 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.924958    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:06:36.934236 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:36.934214    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-65.ec2.internal\" not found"
Jul 14 11:06:36.938605 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:36.938590    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:06:37.023154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.023113    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:37.044353 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.044291    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 11:06:37.046173 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.046159    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"
Jul 14 11:06:37.054933 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.054916    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 11:06:37.276059 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.276031    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:06:37.653891 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.653856    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:06:37.703520 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.703497    2563 apiserver.go:52] "Watching apiserver"
Jul 14 11:06:37.710676 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.710646    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 11:06:37.712339 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.712303    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp","openshift-cluster-node-tuning-operator/tuned-ghmrn","openshift-multus/multus-additional-cni-plugins-xq2pp","openshift-multus/network-metrics-daemon-xlw8z","openshift-network-operator/iptables-alerter-9llnf","kube-system/konnectivity-agent-mc56f","openshift-dns/node-resolver-q65sx","openshift-image-registry/node-ca-zxxqq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal","openshift-multus/multus-ptnk6","openshift-network-diagnostics/network-check-target-k24x7","openshift-ovn-kubernetes/ovnkube-node-5675s","kube-system/global-pull-secret-syncer-tb8l9","kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal"]
Jul 14 11:06:37.714844 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.714820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.716943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.716927    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.717015 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.716985    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:37.717624 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.717599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.717746 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.717627    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-46zxz\""
Jul 14 11:06:37.717746 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.717665    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.719165 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.719142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.721468 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.721445    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-snbgk\""
Jul 14 11:06:37.721542 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.721497    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:37.721603 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.721560    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:37.721719 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.721698    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 11:06:37.721868 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.721846    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 11:06:37.721971 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.721879    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 11:06:37.722038 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.721984    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.722090 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.722050    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.723304 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.723284    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 11:06:37.724143 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.724101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.726260 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.726243    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.726491 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.726471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.727562 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.727502    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.727984 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.727961    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 11:06:37.728085 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.728052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rbrh7\""
Jul 14 11:06:37.729315 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.729292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 11:06:37.729539 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.729519    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p74nz\""
Jul 14 11:06:37.729620 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.729578    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 11:06:37.731178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.731078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.733464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.733440    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.733464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.733456    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 11:06:37.733638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.733468    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.733638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.733584    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gqfpc\""
Jul 14 11:06:37.734373 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.734351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.734863 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.734842    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 11:06:37.737358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.737324    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 11:06:37.737358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.737443    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.737358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.737452    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.737358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.737500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.737358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.737527    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2cmcm\""
Jul 14 11:06:37.738870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.738850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.738962 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.738876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:37.738962 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.738893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vsd\" (UniqueName: \"kubernetes.io/projected/558289b9-aa16-4008-a8fd-4eaa7271e009-kube-api-access-q8vsd\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:37.738962 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.738917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-host-slash\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.739136 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.738995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-system-cni-dir\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739136 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btfrg\" (UniqueName: \"kubernetes.io/projected/61c5aaaa-3e17-47b2-8612-762c7c676f92-kube-api-access-btfrg\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739136 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-hosts-file\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.739136 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3e6c755-ee41-48b9-81d4-7caf68a60441-dbus\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-cnibin\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28bwr\" (UniqueName: \"kubernetes.io/projected/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-kube-api-access-28bwr\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-tmp-dir\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq2kl\" (UniqueName: \"kubernetes.io/projected/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-kube-api-access-cq2kl\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739693 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-iptables-alerter-script\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.739693 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3e6c755-ee41-48b9-81d4-7caf68a60441-kubelet-config\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.739693 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-os-release\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739693 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-cni-binary-copy\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.739693 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.739599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.740046 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.740029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pxxgl\""
Jul 14 11:06:37.740115 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.740079    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.740200 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.740181    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.741805 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.741784    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 11:06:37.741977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.741960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:37.742065 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.742032    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:37.742342 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.742323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xqss4\""
Jul 14 11:06:37.744709 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.744667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.747189 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.747173    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 11:06:37.747289 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.747180    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 11:06:37.747878 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.747769    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 11:06:37.747878 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.747845    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 11:06:37.748401 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.747773    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x24zx\""
Jul 14 11:06:37.748506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.748074    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 11:06:37.748571 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.748130    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 11:06:37.766611 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.766589    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tksg9"
Jul 14 11:06:37.773118 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.773098    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tksg9"
Jul 14 11:06:37.823765 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.823712    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 11:06:37.824523 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.824475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal" event={"ID":"979d7968c8497a3b8cb9fe00c5e6ba4c","Type":"ContainerStarted","Data":"200de1858ba8f34b6623c02e6914ff8f1079b0cd9652fa03cb14b34637f69fe2"}
Jul 14 11:06:37.826021 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.825992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal" event={"ID":"ace48c1090b8dbb21683e27f4527c23e","Type":"ContainerStarted","Data":"80c4a9911c262515e3eab9a151f7dca0073af8c4ca0a829792dc3f0d1b9ec4b7"}
Jul 14 11:06:37.839794 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-cni-binary-copy\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.839918 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vsd\" (UniqueName: \"kubernetes.io/projected/558289b9-aa16-4008-a8fd-4eaa7271e009-kube-api-access-q8vsd\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:37.839918 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-host-slash\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.839918 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839868    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0240294b-4df7-474a-a8f3-9c97f5379cb1-konnectivity-ca\") pod \"konnectivity-agent-mc56f\" (UID: \"0240294b-4df7-474a-a8f3-9c97f5379cb1\") " pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.839918 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-daemon-config\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-run-netns\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.839993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-host-slash\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-ovnkube-script-lib\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-modprobe-d\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysctl-d\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-tmp\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.840139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq2kl\" (UniqueName: \"kubernetes.io/projected/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-kube-api-access-cq2kl\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-iptables-alerter-script\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th8m7\" (UniqueName: \"kubernetes.io/projected/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-kube-api-access-th8m7\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-device-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-kubernetes\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-var-lib-kubelet\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-multus-certs\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-kubelet\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-slash\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsh9d\" (UniqueName: \"kubernetes.io/projected/9189997e-9122-47b3-9ff8-d3baa9707803-kube-api-access-dsh9d\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.840499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-tuned\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-k8s-cni-cncf-io\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-var-lib-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-etc-selinux\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-cni-binary-copy\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-lib-modules\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g22l\" (UniqueName: \"kubernetes.io/projected/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-kube-api-access-2g22l\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3e6c755-ee41-48b9-81d4-7caf68a60441-dbus\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28bwr\" (UniqueName: \"kubernetes.io/projected/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-kube-api-access-28bwr\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0240294b-4df7-474a-a8f3-9c97f5379cb1-agent-certs\") pod \"konnectivity-agent-mc56f\" (UID: \"0240294b-4df7-474a-a8f3-9c97f5379cb1\") " pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-netns\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-cni-multus\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-log-socket\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grzpg\" (UniqueName: \"kubernetes.io/projected/47ba1577-742a-426c-ad17-aecc7e57e72e-kube-api-access-grzpg\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-socket-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-iptables-alerter-script\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-etc-kubernetes\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.841219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3e6c755-ee41-48b9-81d4-7caf68a60441-dbus\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.840941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-systemd\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-systemd\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-cnibin\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3e6c755-ee41-48b9-81d4-7caf68a60441-kubelet-config\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-os-release\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3e6c755-ee41-48b9-81d4-7caf68a60441-kubelet-config\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-system-cni-dir\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btfrg\" (UniqueName: \"kubernetes.io/projected/61c5aaaa-3e17-47b2-8612-762c7c676f92-kube-api-access-btfrg\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-system-cni-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-cni-bin\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-os-release\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-hostroot\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-system-cni-dir\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842094 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.841446    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.841515    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:38.341493208 +0000 UTC m=+2.924032216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-run\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-os-release\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-socket-dir-parent\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-conf-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-systemd-units\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-etc-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.841875    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47ba1577-742a-426c-ad17-aecc7e57e72e-ovn-node-metrics-cert\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.841944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:38.341925918 +0000 UTC m=+2.924464926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.841968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2abf4d3e-c2ff-458c-beb2-f45063b7a331-serviceca\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-tmp-dir\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-ovn\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-cni-bin\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.842977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-env-overrides\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4frnf\" (UniqueName: \"kubernetes.io/projected/2abf4d3e-c2ff-458c-beb2-f45063b7a331-kube-api-access-4frnf\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-sys-fs\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysconfig\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-host\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-cni-binary-copy\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-node-log\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-registration-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-sys\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-kubelet\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-tmp-dir\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-ovnkube-config\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2abf4d3e-c2ff-458c-beb2-f45063b7a331-host\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-kubelet-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-hosts-file\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-cnibin\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.843677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61c5aaaa-3e17-47b2-8612-762c7c676f92-cnibin\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.844574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-cni-netd\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.844574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-cni-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.844574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842647    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-hosts-file\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.844574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-run-ovn-kubernetes\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.844574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysctl-conf\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.844574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.842939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61c5aaaa-3e17-47b2-8612-762c7c676f92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.848779 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.848711    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 11:06:37.852943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.852839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btfrg\" (UniqueName: \"kubernetes.io/projected/61c5aaaa-3e17-47b2-8612-762c7c676f92-kube-api-access-btfrg\") pod \"multus-additional-cni-plugins-xq2pp\" (UID: \"61c5aaaa-3e17-47b2-8612-762c7c676f92\") " pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:37.852943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.852839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vsd\" (UniqueName: \"kubernetes.io/projected/558289b9-aa16-4008-a8fd-4eaa7271e009-kube-api-access-q8vsd\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:37.853251 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.853233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28bwr\" (UniqueName: \"kubernetes.io/projected/9c4afe0b-8c38-448b-a825-cc5bb66e5fd9-kube-api-access-28bwr\") pod \"iptables-alerter-9llnf\" (UID: \"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9\") " pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:37.853336 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.853312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq2kl\" (UniqueName: \"kubernetes.io/projected/0d5a1d51-4bae-4c57-87f8-46ccd09e8d52-kube-api-access-cq2kl\") pod \"node-resolver-q65sx\" (UID: \"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52\") " pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:37.943506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsh9d\" (UniqueName: \"kubernetes.io/projected/9189997e-9122-47b3-9ff8-d3baa9707803-kube-api-access-dsh9d\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.943506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-tuned\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.943506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-k8s-cni-cncf-io\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-var-lib-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-etc-selinux\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-lib-modules\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g22l\" (UniqueName: \"kubernetes.io/projected/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-kube-api-access-2g22l\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0240294b-4df7-474a-a8f3-9c97f5379cb1-agent-certs\") pod \"konnectivity-agent-mc56f\" (UID: \"0240294b-4df7-474a-a8f3-9c97f5379cb1\") " pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-netns\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-cni-multus\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-log-socket\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grzpg\" (UniqueName: \"kubernetes.io/projected/47ba1577-742a-426c-ad17-aecc7e57e72e-kube-api-access-grzpg\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-socket-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-etc-kubernetes\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-systemd\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.943834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-systemd\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-cnibin\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-log-socket\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-system-cni-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-cni-bin\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-system-cni-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.943981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-hostroot\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-hostroot\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-run\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-cni-multus\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-os-release\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-cni-bin\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-socket-dir-parent\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-conf-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-systemd-units\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-etc-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47ba1577-742a-426c-ad17-aecc7e57e72e-ovn-node-metrics-cert\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.944440 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2abf4d3e-c2ff-458c-beb2-f45063b7a331-serviceca\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-ovn\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-cni-bin\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-env-overrides\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4frnf\" (UniqueName: \"kubernetes.io/projected/2abf4d3e-c2ff-458c-beb2-f45063b7a331-kube-api-access-4frnf\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-sys-fs\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysconfig\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-host\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-cni-binary-copy\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-node-log\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-registration-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-sys\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-kubelet\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-ovnkube-config\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2abf4d3e-c2ff-458c-beb2-f45063b7a331-host\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-kubelet-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-var-lib-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-cni-netd\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.945403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-k8s-cni-cncf-io\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-cni-netd\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-cni-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-run-ovn-kubernetes\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysctl-conf\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0240294b-4df7-474a-a8f3-9c97f5379cb1-konnectivity-ca\") pod \"konnectivity-agent-mc56f\" (UID: \"0240294b-4df7-474a-a8f3-9c97f5379cb1\") " pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-daemon-config\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.944990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-run-netns\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-socket-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-etc-kubernetes\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-systemd\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-systemd\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-cnibin\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0240294b-4df7-474a-a8f3-9c97f5379cb1-konnectivity-ca\") pod \"konnectivity-agent-mc56f\" (UID: \"0240294b-4df7-474a-a8f3-9c97f5379cb1\") " pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.947572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-ovnkube-script-lib\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-modprobe-d\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-run\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysctl-d\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-daemon-config\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-tmp\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysctl-d\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-socket-dir-parent\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945711    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-cni-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-etc-selinux\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-run-ovn-kubernetes\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-multus-conf-dir\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-netns\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th8m7\" (UniqueName: \"kubernetes.io/projected/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-kube-api-access-th8m7\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-run-netns\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-device-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-node-log\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.948492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-modprobe-d\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-kubernetes\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-etc-openvswitch\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-var-lib-kubelet\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945927    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-var-lib-kubelet\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-multus-certs\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.945997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-kubelet\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-lib-modules\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-slash\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-cni-bin\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2abf4d3e-c2ff-458c-beb2-f45063b7a331-host\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-run-ovn\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946139    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-kubernetes\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-run-multus-certs\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-device-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946197    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-slash\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2abf4d3e-c2ff-458c-beb2-f45063b7a331-serviceca\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-host-kubelet\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.949394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysconfig\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-ovnkube-script-lib\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-os-release\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-cni-binary-copy\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-env-overrides\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-kubelet-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-sys-fs\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47ba1577-742a-426c-ad17-aecc7e57e72e-ovnkube-config\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-host\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9189997e-9122-47b3-9ff8-d3baa9707803-registration-dir\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-sys\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-host-var-lib-kubelet\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47ba1577-742a-426c-ad17-aecc7e57e72e-systemd-units\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.946962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-sysctl-conf\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.947206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0240294b-4df7-474a-a8f3-9c97f5379cb1-agent-certs\") pod \"konnectivity-agent-mc56f\" (UID: \"0240294b-4df7-474a-a8f3-9c97f5379cb1\") " pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.948126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-etc-tuned\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.949162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47ba1577-742a-426c-ad17-aecc7e57e72e-ovn-node-metrics-cert\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.950365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.949188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-tmp\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.952679 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.952357    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:06:37.952679 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.952383    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:06:37.952679 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.952398    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:37.952679 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:37.952469    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:38.452451129 +0000 UTC m=+3.034990129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:37.953645 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.953618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsh9d\" (UniqueName: \"kubernetes.io/projected/9189997e-9122-47b3-9ff8-d3baa9707803-kube-api-access-dsh9d\") pod \"aws-ebs-csi-driver-node-26vwp\" (UID: \"9189997e-9122-47b3-9ff8-d3baa9707803\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:37.955332 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.955307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grzpg\" (UniqueName: \"kubernetes.io/projected/47ba1577-742a-426c-ad17-aecc7e57e72e-kube-api-access-grzpg\") pod \"ovnkube-node-5675s\" (UID: \"47ba1577-742a-426c-ad17-aecc7e57e72e\") " pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:37.955779 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.955758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g22l\" (UniqueName: \"kubernetes.io/projected/22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e-kube-api-access-2g22l\") pod \"tuned-ghmrn\" (UID: \"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e\") " pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:37.956720 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.956692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th8m7\" (UniqueName: \"kubernetes.io/projected/c0aeb76c-c70d-4f63-bd32-3be1438cd2e3-kube-api-access-th8m7\") pod \"multus-ptnk6\" (UID: \"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3\") " pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:37.957486 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:37.957468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4frnf\" (UniqueName: \"kubernetes.io/projected/2abf4d3e-c2ff-458c-beb2-f45063b7a331-kube-api-access-4frnf\") pod \"node-ca-zxxqq\" (UID: \"2abf4d3e-c2ff-458c-beb2-f45063b7a331\") " pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:38.025105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.025076    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q65sx"
Jul 14 11:06:38.033123 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.033097    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xq2pp"
Jul 14 11:06:38.042136 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.042114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9llnf"
Jul 14 11:06:38.049088 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.049070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:06:38.056223 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.056199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zxxqq"
Jul 14 11:06:38.065099 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.065066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp"
Jul 14 11:06:38.073935 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.073914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ghmrn"
Jul 14 11:06:38.082743 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.082720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ptnk6"
Jul 14 11:06:38.088685 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.088654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:06:38.349879 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.349787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:38.349879 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.349833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:38.350122 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.349954    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:38.350122 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.349963    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:38.350122 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.350023    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:39.349996607 +0000 UTC m=+3.932535593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:38.350122 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.350043    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:39.350032707 +0000 UTC m=+3.932571699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:38.551813 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.551768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:38.551994 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.551974    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:06:38.552043 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.552002    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:06:38.552043 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.552018    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:38.552132 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.552092    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:39.552070893 +0000 UTC m=+4.134609904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:38.777324 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.777230    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:01:38 +0000 UTC" deadline="2028-04-03 07:43:48.111826648 +0000 UTC"
Jul 14 11:06:38.777324 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.777290    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15092h37m9.334553099s"
Jul 14 11:06:38.819907 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:38.819876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:38.820040 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:38.819993    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:38.845689 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.845635    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ecb3cb_76e8_4f58_8c7a_f30d9409dd2e.slice/crio-792ab32217eee2c5b2864c15d22ec542e4b4441b2df9ebea10065d644eb54b0f WatchSource:0}: Error finding container 792ab32217eee2c5b2864c15d22ec542e4b4441b2df9ebea10065d644eb54b0f: Status 404 returned error can't find the container with id 792ab32217eee2c5b2864c15d22ec542e4b4441b2df9ebea10065d644eb54b0f
Jul 14 11:06:38.849042 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.849012    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ba1577_742a_426c_ad17_aecc7e57e72e.slice/crio-63632a5100f6305bb2d5667376f660afc91a878f020ceb292c7a7ce8283fed45 WatchSource:0}: Error finding container 63632a5100f6305bb2d5667376f660afc91a878f020ceb292c7a7ce8283fed45: Status 404 returned error can't find the container with id 63632a5100f6305bb2d5667376f660afc91a878f020ceb292c7a7ce8283fed45
Jul 14 11:06:38.851776 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.851720    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4afe0b_8c38_448b_a825_cc5bb66e5fd9.slice/crio-1712996a34c15ee3c55062ff0fc27f9b5a4f9320b30ef69a2dfc4d3494cf8767 WatchSource:0}: Error finding container 1712996a34c15ee3c55062ff0fc27f9b5a4f9320b30ef69a2dfc4d3494cf8767: Status 404 returned error can't find the container with id 1712996a34c15ee3c55062ff0fc27f9b5a4f9320b30ef69a2dfc4d3494cf8767
Jul 14 11:06:38.852599 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.852561    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2abf4d3e_c2ff_458c_beb2_f45063b7a331.slice/crio-fcd50c0c70669398e5afbf7fa10d35f22014ea61a33c7893a627a9c148f38801 WatchSource:0}: Error finding container fcd50c0c70669398e5afbf7fa10d35f22014ea61a33c7893a627a9c148f38801: Status 404 returned error can't find the container with id fcd50c0c70669398e5afbf7fa10d35f22014ea61a33c7893a627a9c148f38801
Jul 14 11:06:38.853847 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.853553    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9189997e_9122_47b3_9ff8_d3baa9707803.slice/crio-15ed1e3c968b45fce59597540aaed2775c90b260d7d39970823e22562b664d64 WatchSource:0}: Error finding container 15ed1e3c968b45fce59597540aaed2775c90b260d7d39970823e22562b664d64: Status 404 returned error can't find the container with id 15ed1e3c968b45fce59597540aaed2775c90b260d7d39970823e22562b664d64
Jul 14 11:06:38.855814 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.855726    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5a1d51_4bae_4c57_87f8_46ccd09e8d52.slice/crio-04f76ec5192e9ed907175105fea4ecabf5b11e72f5a100a53947cd9e6be90df7 WatchSource:0}: Error finding container 04f76ec5192e9ed907175105fea4ecabf5b11e72f5a100a53947cd9e6be90df7: Status 404 returned error can't find the container with id 04f76ec5192e9ed907175105fea4ecabf5b11e72f5a100a53947cd9e6be90df7
Jul 14 11:06:38.879392 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.879365    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0aeb76c_c70d_4f63_bd32_3be1438cd2e3.slice/crio-510afb334819c9d8262e370c5afe2cca7940e2def472fc439fa06c2ce8386326 WatchSource:0}: Error finding container 510afb334819c9d8262e370c5afe2cca7940e2def472fc439fa06c2ce8386326: Status 404 returned error can't find the container with id 510afb334819c9d8262e370c5afe2cca7940e2def472fc439fa06c2ce8386326
Jul 14 11:06:38.880396 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:06:38.880376    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c5aaaa_3e17_47b2_8612_762c7c676f92.slice/crio-614c3ad6258d11e9a023a1a4d382daeb9681dd8d566e4529546f6340066a5efd WatchSource:0}: Error finding container 614c3ad6258d11e9a023a1a4d382daeb9681dd8d566e4529546f6340066a5efd: Status 404 returned error can't find the container with id 614c3ad6258d11e9a023a1a4d382daeb9681dd8d566e4529546f6340066a5efd
Jul 14 11:06:39.358051 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.357759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:39.358210 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.358088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:39.358275 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.358218    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:39.358335 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.358282    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:41.358262003 +0000 UTC m=+5.940800995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:39.358814 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.358793    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:39.358902 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.358863    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:41.358841977 +0000 UTC m=+5.941380974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:39.559284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.559232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:39.559503 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.559483    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:06:39.559603 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.559510    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:06:39.559603 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.559525    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:39.559603 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.559591    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:41.559570441 +0000 UTC m=+6.142109429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:39.777560 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.777414    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:01:38 +0000 UTC" deadline="2028-02-10 06:04:23.936632942 +0000 UTC"
Jul 14 11:06:39.777560 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.777455    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13818h57m44.159182905s"
Jul 14 11:06:39.824972 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.822013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:39.824972 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.822145    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:39.824972 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.824803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:39.824972 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:39.824928    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:39.842086 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.842044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ghmrn" event={"ID":"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e","Type":"ContainerStarted","Data":"792ab32217eee2c5b2864c15d22ec542e4b4441b2df9ebea10065d644eb54b0f"}
Jul 14 11:06:39.849193 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.849163    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerStarted","Data":"614c3ad6258d11e9a023a1a4d382daeb9681dd8d566e4529546f6340066a5efd"}
Jul 14 11:06:39.855289 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.855227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ptnk6" event={"ID":"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3","Type":"ContainerStarted","Data":"510afb334819c9d8262e370c5afe2cca7940e2def472fc439fa06c2ce8386326"}
Jul 14 11:06:39.868930 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.868846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q65sx" event={"ID":"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52","Type":"ContainerStarted","Data":"04f76ec5192e9ed907175105fea4ecabf5b11e72f5a100a53947cd9e6be90df7"}
Jul 14 11:06:39.882190 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.882102    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"63632a5100f6305bb2d5667376f660afc91a878f020ceb292c7a7ce8283fed45"}
Jul 14 11:06:39.889478 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.889404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mc56f" event={"ID":"0240294b-4df7-474a-a8f3-9c97f5379cb1","Type":"ContainerStarted","Data":"1faf6721e9d84a011bdb657e33f03771359dce9252bde5d5583baf9b45ab7c7c"}
Jul 14 11:06:39.901954 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.901928    2563 generic.go:356] "Generic (PLEG): container finished" podID="979d7968c8497a3b8cb9fe00c5e6ba4c" containerID="98de961512061b1957708e061e5a26687306f916d70aa4e9169210f1718da5bb" exitCode=0
Jul 14 11:06:39.902049 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.902005    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal" event={"ID":"979d7968c8497a3b8cb9fe00c5e6ba4c","Type":"ContainerDied","Data":"98de961512061b1957708e061e5a26687306f916d70aa4e9169210f1718da5bb"}
Jul 14 11:06:39.909153 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.908590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal" event={"ID":"ace48c1090b8dbb21683e27f4527c23e","Type":"ContainerStarted","Data":"8efda5a7c27624ccd2a2f1f7f2e52ad82ca298df72a9db4291965a5631426a52"}
Jul 14 11:06:39.916518 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.916418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp" event={"ID":"9189997e-9122-47b3-9ff8-d3baa9707803","Type":"ContainerStarted","Data":"15ed1e3c968b45fce59597540aaed2775c90b260d7d39970823e22562b664d64"}
Jul 14 11:06:39.924772 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.923898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9llnf" event={"ID":"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9","Type":"ContainerStarted","Data":"1712996a34c15ee3c55062ff0fc27f9b5a4f9320b30ef69a2dfc4d3494cf8767"}
Jul 14 11:06:39.927747 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:39.927502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zxxqq" event={"ID":"2abf4d3e-c2ff-458c-beb2-f45063b7a331","Type":"ContainerStarted","Data":"fcd50c0c70669398e5afbf7fa10d35f22014ea61a33c7893a627a9c148f38801"}
Jul 14 11:06:40.819944 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:40.819909    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:40.820466 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:40.820054    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:40.943131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:40.943088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal" event={"ID":"979d7968c8497a3b8cb9fe00c5e6ba4c","Type":"ContainerStarted","Data":"03d3948e99f8075857a7fee89b26a20fcaec283c1e844a55ecb27f2e122c1608"}
Jul 14 11:06:40.960130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:40.960071    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-65.ec2.internal" podStartSLOduration=3.960050646 podStartE2EDuration="3.960050646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:39.942098994 +0000 UTC m=+4.524638004" watchObservedRunningTime="2026-07-14 11:06:40.960050646 +0000 UTC m=+5.542589656"
Jul 14 11:06:41.377631 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:41.377599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:41.377835 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:41.377650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:41.377894 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.377834    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:41.377894 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.377893    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:45.377875507 +0000 UTC m=+9.960414493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:41.378288 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.378271    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:41.378388 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.378323    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:45.378307081 +0000 UTC m=+9.960846075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:41.580055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:41.580020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:41.580225 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.580189    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:06:41.580225 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.580210    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:06:41.580225 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.580222    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:41.580395 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.580280    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:45.580262373 +0000 UTC m=+10.162801372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:41.821268 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:41.820776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:41.821268 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:41.820781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:41.821268 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.820899    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:41.821268 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:41.820989    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:42.820367 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:42.819892    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:42.820367 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:42.820029    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:43.819973 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:43.819941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:43.820410 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:43.820092    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:43.820525 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:43.820507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:43.820637 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:43.820611    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:44.820117 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:44.820076    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:44.820520 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:44.820219    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:45.412072 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:45.411973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:45.412072 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:45.412022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:45.412352 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.412135    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:45.412352 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.412182    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:45.412352 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.412212    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:53.412189822 +0000 UTC m=+17.994728835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:45.412352 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.412234    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:53.412223952 +0000 UTC m=+17.994762938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:45.613943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:45.613909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:45.614147 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.614115    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:06:45.614147 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.614135    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:06:45.614147 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.614147    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:45.614355 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.614201    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:53.61418486 +0000 UTC m=+18.196723848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:45.821147 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:45.820447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:45.821147 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.820546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:45.821147 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:45.820942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:45.821147 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:45.821052    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:46.820089 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:46.820050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:46.820273 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:46.820182    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:47.820684 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:47.820208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:47.820684 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:47.820349    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:47.820684 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:47.820405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:47.820684 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:47.820541    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:48.819709 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:48.819669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:48.819888 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:48.819803    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:49.819870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:49.819844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:49.820149 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:49.819947    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:49.820149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:49.820004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:49.820149 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:49.820106    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:50.819484 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:50.819453    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:50.819646 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:50.819590    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:51.820277 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:51.820243    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:51.820750 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:51.820377    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:51.820750 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:51.820427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:51.820750 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:51.820534    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:52.819724 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:52.819680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:52.819960 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:52.819801    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:53.473360 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:53.473331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:53.473360 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:53.473366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:53.473751 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.473455    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:53.473751 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.473471    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:53.473751 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.473504    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:09.473490626 +0000 UTC m=+34.056029609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:06:53.473751 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.473523    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:09.47350955 +0000 UTC m=+34.056048532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:06:53.675222 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:53.675190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:53.675427 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.675342    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:06:53.675427 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.675365    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:06:53.675427 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.675376    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:53.675573 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.675436    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:09.675420405 +0000 UTC m=+34.257959388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:06:53.819492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:53.819416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:53.819677 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.819544    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:53.819677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:53.819613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:53.819794 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:53.819743    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:54.819422 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:54.819390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:54.819802 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:54.819500    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:55.820058 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:55.820026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:55.820457 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:55.820126    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:55.820457 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:55.820191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:55.820457 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:55.820310    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:56.819841 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:56.819810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:56.819999 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:56.819924    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:57.827176 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.827149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:57.827550 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:57.827270    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:57.827638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.827616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:57.827765 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:57.827742    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:06:57.979877 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.979844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp" event={"ID":"9189997e-9122-47b3-9ff8-d3baa9707803","Type":"ContainerStarted","Data":"88ed703a3c655c3c851d8559567619968d9715092679666077953ad6c5749c64"}
Jul 14 11:06:57.981572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.981545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zxxqq" event={"ID":"2abf4d3e-c2ff-458c-beb2-f45063b7a331","Type":"ContainerStarted","Data":"f72cb545c9008777e1c08f1fe50ddfeacfa7c4293c95185bf67bba038e2188b9"}
Jul 14 11:06:57.982962 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.982942    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ghmrn" event={"ID":"22ecb3cb-76e8-4f58-8c7a-f30d9409dd2e","Type":"ContainerStarted","Data":"20bbf942f03fb65795e48c881ce7b59952ab4d5412ad32c3285b7b9d5d5eb06a"}
Jul 14 11:06:57.984462 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.984437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerStarted","Data":"07c6c00d15b3600b7694c2e28f237cf8026af96dca983e54f168350b86836cae"}
Jul 14 11:06:57.985933 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.985912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ptnk6" event={"ID":"c0aeb76c-c70d-4f63-bd32-3be1438cd2e3","Type":"ContainerStarted","Data":"0c5b07e95ce29c4f94f589cdc0c512d219d57048fb51e27eb04795f3c6a5c3cf"}
Jul 14 11:06:57.987496 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.987401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q65sx" event={"ID":"0d5a1d51-4bae-4c57-87f8-46ccd09e8d52","Type":"ContainerStarted","Data":"09447594873f92365bac33a04d01faacb9e0b34ab7ae404e80e234dc657603ca"}
Jul 14 11:06:57.988757 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.988728    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"9c466326538dd3c8c2adf6e9a55e74d79a0f48b402b217611bf0e45c11f2674d"}
Jul 14 11:06:57.989951 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:57.989931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mc56f" event={"ID":"0240294b-4df7-474a-a8f3-9c97f5379cb1","Type":"ContainerStarted","Data":"dd2e2402295cacfea515c9dc7e0ec3afe39f00c4624c1f3a2071ce22d256a12f"}
Jul 14 11:06:58.000387 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.000348    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zxxqq" podStartSLOduration=3.135428942 podStartE2EDuration="22.000317825s" totalImagesPullingTime="18.864888883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:57.999894972 +0000 UTC m=+22.582433977" watchObservedRunningTime="2026-07-14 11:06:58.000317825 +0000 UTC m=+22.582856831"
Jul 14 11:06:58.000477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.000419    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-65.ec2.internal" podStartSLOduration=21.000413717 podStartE2EDuration="21.000413717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:40.960767012 +0000 UTC m=+5.543306040" watchObservedRunningTime="2026-07-14 11:06:58.000413717 +0000 UTC m=+22.582952721"
Jul 14 11:06:58.023533 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.023492    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ghmrn" podStartSLOduration=3.140240083 podStartE2EDuration="22.023477793s" totalImagesPullingTime="18.88323771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:58.022848969 +0000 UTC m=+22.605387974" watchObservedRunningTime="2026-07-14 11:06:58.023477793 +0000 UTC m=+22.606016799"
Jul 14 11:06:58.089267 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.089212    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ptnk6" podStartSLOduration=3.204895765 podStartE2EDuration="22.089190883s" totalImagesPullingTime="18.884295118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:58.060112804 +0000 UTC m=+22.642651809" watchObservedRunningTime="2026-07-14 11:06:58.089190883 +0000 UTC m=+22.671729891"
Jul 14 11:06:58.134915 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.134869    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q65sx" podStartSLOduration=3.293362945 podStartE2EDuration="22.134850266s" totalImagesPullingTime="18.841487321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:58.090145155 +0000 UTC m=+22.672684159" watchObservedRunningTime="2026-07-14 11:06:58.134850266 +0000 UTC m=+22.717389271"
Jul 14 11:06:58.820129 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.819911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:06:58.820290 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:58.820225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:06:58.996148 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996076    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:06:58.997016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996535    2563 generic.go:356] "Generic (PLEG): container finished" podID="47ba1577-742a-426c-ad17-aecc7e57e72e" containerID="92cb93323f75971561fdb52abe9cbf0711575909849cd6913799c0729208c1c0" exitCode=1
Jul 14 11:06:58.997016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"639cad9ffea2876206f3e1173186dbebbf9a09d8ef40ed93794f222b523e8b3c"}
Jul 14 11:06:58.997016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"351dfa7c18fba968c5c586afb2135bac742595c94ef9bef927eb0400f0db2649"}
Jul 14 11:06:58.997016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996790    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"6dd7899550a2d1581e484a914b29794eaff3e3d905ae95a016e8eb80e0e43eda"}
Jul 14 11:06:58.997016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"f4552a5995dc89cbb6259da0574509c83d0eb6dd8ba05f9ae8c2664012a2f9bf"}
Jul 14 11:06:58.997016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.996882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerDied","Data":"92cb93323f75971561fdb52abe9cbf0711575909849cd6913799c0729208c1c0"}
Jul 14 11:06:58.998438 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:58.998405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9llnf" event={"ID":"9c4afe0b-8c38-448b-a825-cc5bb66e5fd9","Type":"ContainerStarted","Data":"c32a8b307421923d654771dc2dd46184dae43d8dc77f75ca471afe982709d5dc"}
Jul 14 11:06:59.000336 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.000311    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c5aaaa-3e17-47b2-8612-762c7c676f92" containerID="07c6c00d15b3600b7694c2e28f237cf8026af96dca983e54f168350b86836cae" exitCode=0
Jul 14 11:06:59.000449 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.000422    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerDied","Data":"07c6c00d15b3600b7694c2e28f237cf8026af96dca983e54f168350b86836cae"}
Jul 14 11:06:59.016040 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.016005    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9llnf" podStartSLOduration=4.152782064 podStartE2EDuration="23.015992396s" totalImagesPullingTime="18.863210332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:59.015891936 +0000 UTC m=+23.598430974" watchObservedRunningTime="2026-07-14 11:06:59.015992396 +0000 UTC m=+23.598531401"
Jul 14 11:06:59.019157 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.016486    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mc56f" podStartSLOduration=4.147606231 podStartE2EDuration="23.016473184s" totalImagesPullingTime="18.868866953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:58.17714655 +0000 UTC m=+22.759685555" watchObservedRunningTime="2026-07-14 11:06:59.016473184 +0000 UTC m=+23.599012169"
Jul 14 11:06:59.441286 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.441215    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 11:06:59.793588 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.793454    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T11:06:59.441234661Z","UUID":"506f6f28-8d51-4fc8-bc6e-9954c686f784","Handler":null,"Name":"","Endpoint":""}
Jul 14 11:06:59.795308 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.795275    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 11:06:59.795435 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.795319    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 11:06:59.819396 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.819374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:06:59.819523 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:59.819484    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:06:59.819590 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:06:59.819553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:06:59.819707 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:06:59.819686    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:00.004517 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:00.004488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp" event={"ID":"9189997e-9122-47b3-9ff8-d3baa9707803","Type":"ContainerStarted","Data":"1253de7428c6ae783d94ec7ad2e85a7549635b68866b5882f01c9eb08cfe1a19"}
Jul 14 11:07:00.820167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:00.820100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:00.820316 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:00.820199    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:07:01.010845 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:01.010811    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:07:01.011289 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:01.011215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"63d9b42e3e5f386dd3c1410902cb53e41fed22b29baa517305877cf22a6a5fe0"}
Jul 14 11:07:01.820091 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:01.820053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:01.820255 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:01.820060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:01.820255 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:01.820174    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:07:01.820357 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:01.820265    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:02.381431 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:02.381411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:07:02.381899 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:02.381885    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:07:02.383582 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:02.383495    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:07:02.385055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:02.384167    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mc56f"
Jul 14 11:07:02.820119 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:02.819891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:02.820254 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:02.820175    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:07:03.017582 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.017541    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c5aaaa-3e17-47b2-8612-762c7c676f92" containerID="7c6cec1a59e099f1be04fe0f9ffcd450da65f6e11e2b992e4f93553317d86091" exitCode=0
Jul 14 11:07:03.017751 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.017628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerDied","Data":"7c6cec1a59e099f1be04fe0f9ffcd450da65f6e11e2b992e4f93553317d86091"}
Jul 14 11:07:03.021131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.021115    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:07:03.021486 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.021460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"676cfb94bf18c6227a8cfb4b0a4585445a51ebebbd02ca0e0d7b9354db5b4995"}
Jul 14 11:07:03.021708 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.021686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:07:03.021861 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.021761    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:07:03.021861 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.021809    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:07:03.021968 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.021866    2563 scope.go:117] "RemoveContainer" containerID="92cb93323f75971561fdb52abe9cbf0711575909849cd6913799c0729208c1c0"
Jul 14 11:07:03.023991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.023968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp" event={"ID":"9189997e-9122-47b3-9ff8-d3baa9707803","Type":"ContainerStarted","Data":"8ae5e7bd65debe464c5e890aaefe3862553109edaa6819b79830cf4acd30dc25"}
Jul 14 11:07:03.037584 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.037563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:07:03.037828 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.037812    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:07:03.053712 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.053677    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-26vwp" podStartSLOduration=3.958217988 podStartE2EDuration="27.053664783s" totalImagesPullingTime="23.095446795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:03.053139517 +0000 UTC m=+27.635678543" watchObservedRunningTime="2026-07-14 11:07:03.053664783 +0000 UTC m=+27.636203779"
Jul 14 11:07:03.819874 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.819838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:03.820461 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:03.819955    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:07:03.820461 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:03.820007    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:03.820461 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:03.820105    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:04.029786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.029763    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:07:04.030116 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.030092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" event={"ID":"47ba1577-742a-426c-ad17-aecc7e57e72e","Type":"ContainerStarted","Data":"8e35b3ba59796653bb8e7ffe1da0140f0229ed432135f8a226d1e7b9a4df62b6"}
Jul 14 11:07:04.059550 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.059507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5675s" podStartSLOduration=9.118005198 podStartE2EDuration="28.059492797s" totalImagesPullingTime="18.941487599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:04.059159116 +0000 UTC m=+28.641698120" watchObservedRunningTime="2026-07-14 11:07:04.059492797 +0000 UTC m=+28.642031826"
Jul 14 11:07:04.702983 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.702956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tb8l9"]
Jul 14 11:07:04.703179 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.703064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:04.703179 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:04.703146    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:07:04.706450 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.706425    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k24x7"]
Jul 14 11:07:04.706581 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.706515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:04.706649 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:04.706598    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:07:04.707148 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.707124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlw8z"]
Jul 14 11:07:04.707262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:04.707205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:04.707319 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:04.707292    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:05.034892 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:05.034864    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c5aaaa-3e17-47b2-8612-762c7c676f92" containerID="340d44649491bdaa6928982f448d280ba2e6c51187c5504b809adf8a35a65545" exitCode=0
Jul 14 11:07:05.035591 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:05.034949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerDied","Data":"340d44649491bdaa6928982f448d280ba2e6c51187c5504b809adf8a35a65545"}
Jul 14 11:07:05.819985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:05.819957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:05.820214 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:05.820036    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:07:06.819927 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:06.819773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:06.820275 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:06.819821    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:06.820275 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:06.820004    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:06.820275 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:06.820071    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:07:07.042859 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:07.042831    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c5aaaa-3e17-47b2-8612-762c7c676f92" containerID="f5f315bda7b5a62ac835f0834be35da545979727f8ee5e4aa391043ee625b02a" exitCode=0
Jul 14 11:07:07.043012 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:07.042865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerDied","Data":"f5f315bda7b5a62ac835f0834be35da545979727f8ee5e4aa391043ee625b02a"}
Jul 14 11:07:07.822844 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:07.822813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:07.823373 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:07.822944    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:07:08.819961 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:08.819929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:08.820145 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:08.820064    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:07:08.820217 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:08.820164    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:08.820326 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:08.820293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:09.499968 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:09.499930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:09.499968 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:09.499966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:09.500562 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.500069    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:07:09.500562 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.500139    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:07:09.500562 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.500147    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.500124044 +0000 UTC m=+66.082663078 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:07:09.500562 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.500181    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret podName:c3e6c755-ee41-48b9-81d4-7caf68a60441 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.500170103 +0000 UTC m=+66.082709086 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret") pod "global-pull-secret-syncer-tb8l9" (UID: "c3e6c755-ee41-48b9-81d4-7caf68a60441") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:07:09.701389 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:09.701317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:09.701508 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.701443    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:07:09.701508 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.701460    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:07:09.701508 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.701469    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k7wsz for pod openshift-network-diagnostics/network-check-target-k24x7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:07:09.701606 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.701515    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz podName:28f7113a-277a-4ace-9914-c288126f7bd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.701500353 +0000 UTC m=+66.284039349 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7wsz" (UniqueName: "kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz") pod "network-check-target-k24x7" (UID: "28f7113a-277a-4ace-9914-c288126f7bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:07:09.823070 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:09.823044    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:09.823214 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:09.823163    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tb8l9" podUID="c3e6c755-ee41-48b9-81d4-7caf68a60441"
Jul 14 11:07:10.820248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:10.820220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:10.820811 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:10.820335    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k24x7" podUID="28f7113a-277a-4ace-9914-c288126f7bd9"
Jul 14 11:07:10.820811 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:10.820370    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:10.820811 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:10.820453    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlw8z" podUID="558289b9-aa16-4008-a8fd-4eaa7271e009"
Jul 14 11:07:11.284978 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.284953    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-65.ec2.internal" event="NodeReady"
Jul 14 11:07:11.285135 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.285110    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 11:07:11.364033 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.363997    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-657c88457b-mrjlh"]
Jul 14 11:07:11.381467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.381441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.384002 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.383976    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 11:07:11.384398 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.384375    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 11:07:11.385131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.385110    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cn4jk\""
Jul 14 11:07:11.385131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.385114    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 11:07:11.386472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.386058    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-657c88457b-mrjlh"]
Jul 14 11:07:11.386986 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.386961    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xkclt"]
Jul 14 11:07:11.391151 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.390435    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 11:07:11.405465 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.405443    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-htvd2"]
Jul 14 11:07:11.405601 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.405569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.409786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.408931    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 11:07:11.409786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.409455    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 11:07:11.409786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.409704    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 11:07:11.410252 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.410136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7vlft\""
Jul 14 11:07:11.423321 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.423302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xkclt"]
Jul 14 11:07:11.423407 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.423330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-htvd2"]
Jul 14 11:07:11.423455 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.423420    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.425714 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.425690    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-grcgd\""
Jul 14 11:07:11.425814 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.425769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 11:07:11.425814 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.425809    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 11:07:11.514437 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/832fb706-40df-48ff-b0b9-ff498711dc2a-cert\") pod \"ingress-canary-xkclt\" (UID: \"832fb706-40df-48ff-b0b9-ff498711dc2a\") " pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.514437 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-image-registry-private-configuration\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514437 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c9fd828-d26d-4dca-b75a-16f200bec483-ca-trust-extracted\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514731 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-trusted-ca\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514731 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f49s\" (UniqueName: \"kubernetes.io/projected/832fb706-40df-48ff-b0b9-ff498711dc2a-kube-api-access-7f49s\") pod \"ingress-canary-xkclt\" (UID: \"832fb706-40df-48ff-b0b9-ff498711dc2a\") " pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.514731 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514731 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-installation-pull-secrets\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514731 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-certificates\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514983 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9twp\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-kube-api-access-b9twp\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.514983 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.514781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-bound-sa-token\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.546030 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.545993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:11.615841 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.615813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-certificates\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.615841 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.615841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9twp\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-kube-api-access-b9twp\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.615859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-config-volume\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.616025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.615878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-bound-sa-token\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.615895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zbv2\" (UniqueName: \"kubernetes.io/projected/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-kube-api-access-6zbv2\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.616025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.615992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/832fb706-40df-48ff-b0b9-ff498711dc2a-cert\") pod \"ingress-canary-xkclt\" (UID: \"832fb706-40df-48ff-b0b9-ff498711dc2a\") " pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.616206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-image-registry-private-configuration\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c9fd828-d26d-4dca-b75a-16f200bec483-ca-trust-extracted\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-trusted-ca\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.616206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f49s\" (UniqueName: \"kubernetes.io/projected/832fb706-40df-48ff-b0b9-ff498711dc2a-kube-api-access-7f49s\") pod \"ingress-canary-xkclt\" (UID: \"832fb706-40df-48ff-b0b9-ff498711dc2a\") " pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.616206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-tmp-dir\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.616488 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616488 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.616392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-installation-pull-secrets\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.616488 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:11.616452    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:11.616488 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:11.616469    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:11.616707 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:11.616531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:12.116511249 +0000 UTC m=+36.699050249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:11.619960 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.619934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/832fb706-40df-48ff-b0b9-ff498711dc2a-cert\") pod \"ingress-canary-xkclt\" (UID: \"832fb706-40df-48ff-b0b9-ff498711dc2a\") " pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.624548 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.624200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c9fd828-d26d-4dca-b75a-16f200bec483-ca-trust-extracted\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.625055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.624934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-certificates\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.625055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.625012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-installation-pull-secrets\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.625055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.625046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-trusted-ca\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.625299 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.625274    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-image-registry-private-configuration\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.625804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.625785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f49s\" (UniqueName: \"kubernetes.io/projected/832fb706-40df-48ff-b0b9-ff498711dc2a-kube-api-access-7f49s\") pod \"ingress-canary-xkclt\" (UID: \"832fb706-40df-48ff-b0b9-ff498711dc2a\") " pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.625883 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.625867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9twp\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-kube-api-access-b9twp\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.627206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.627188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-bound-sa-token\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:11.642583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.642559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:11.716897 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.716874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-config-volume\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.716989 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.716901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zbv2\" (UniqueName: \"kubernetes.io/projected/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-kube-api-access-6zbv2\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.716989 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.716938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.717101 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:11.717007    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:11.717101 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:11.717053    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:07:12.217041404 +0000 UTC m=+36.799580386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:11.717101 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.717054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-tmp-dir\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.717340 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.717322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-tmp-dir\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.717466 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.717449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-config-volume\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.719892 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.719876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xkclt"
Jul 14 11:07:11.727086 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.727066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zbv2\" (UniqueName: \"kubernetes.io/projected/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-kube-api-access-6zbv2\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:11.787786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.787735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:11.822642 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.822623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:11.825280 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.825264    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 11:07:11.888526 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.888350    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:11.932239 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.932213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xkclt"]
Jul 14 11:07:11.936128 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:07:11.936110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832fb706_40df_48ff_b0b9_ff498711dc2a.slice/crio-f505e0e9e83da452a885ba474fb5a6c318db25d0749e0ba8c24a5b01f1325d78 WatchSource:0}: Error finding container f505e0e9e83da452a885ba474fb5a6c318db25d0749e0ba8c24a5b01f1325d78: Status 404 returned error can't find the container with id f505e0e9e83da452a885ba474fb5a6c318db25d0749e0ba8c24a5b01f1325d78
Jul 14 11:07:11.989511 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:11.989485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:12.056955 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.056881    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c5aaaa-3e17-47b2-8612-762c7c676f92" containerID="e98e1a555abe3aea36f1cd308f9578869c1135f2f9c7ab77b16ee39cafdf989a" exitCode=0
Jul 14 11:07:12.057070 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.056962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerDied","Data":"e98e1a555abe3aea36f1cd308f9578869c1135f2f9c7ab77b16ee39cafdf989a"}
Jul 14 11:07:12.058091 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.058064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xkclt" event={"ID":"832fb706-40df-48ff-b0b9-ff498711dc2a","Type":"ContainerStarted","Data":"f505e0e9e83da452a885ba474fb5a6c318db25d0749e0ba8c24a5b01f1325d78"}
Jul 14 11:07:12.120576 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.120556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:12.120745 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:12.120726    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:12.120803 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:12.120751    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:12.120836 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:12.120816    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:13.120792408 +0000 UTC m=+37.703331411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:12.221125 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.221101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:12.221311 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:12.221291    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:12.221388 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:12.221376    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:07:13.221350092 +0000 UTC m=+37.803889078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:12.747682 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.747638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:12.820279 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.820253    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:12.820452 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.820434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:12.823130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.822871    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 11:07:12.823130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.822961    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 11:07:12.823130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.822968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zjjh4\""
Jul 14 11:07:12.823130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.823062    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhbfh\""
Jul 14 11:07:12.823758 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.823252    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 11:07:12.848427 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:12.848397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:13.063846 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:13.063807    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c5aaaa-3e17-47b2-8612-762c7c676f92" containerID="82e5dbf8eefccfd4702e85d0bcef22084b1e2d495358e773b6c8b3d72797e1a9" exitCode=0
Jul 14 11:07:13.063998 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:13.063892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerDied","Data":"82e5dbf8eefccfd4702e85d0bcef22084b1e2d495358e773b6c8b3d72797e1a9"}
Jul 14 11:07:13.128630 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:13.128545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:13.128757 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:13.128720    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:13.128757 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:13.128741    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:13.128851 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:13.128811    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:15.128790231 +0000 UTC m=+39.711329230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:13.229551 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:13.229510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:13.229699 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:13.229615    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:13.229699 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:13.229695    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:07:15.229675262 +0000 UTC m=+39.812214259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:13.765733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:13.765444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:13.952704 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:13.952664    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:14.070471 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:14.070438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" event={"ID":"61c5aaaa-3e17-47b2-8612-762c7c676f92","Type":"ContainerStarted","Data":"a8769127490aa9420ccd2bd896f632b9f7f6ff6738effab31010eeab8f9ef263"}
Jul 14 11:07:14.093275 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:14.093231    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xq2pp" podStartSLOduration=7.777609341 podStartE2EDuration="38.093213585s" totalImagesPullingTime="30.315604244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:14.092920567 +0000 UTC m=+38.675459608" watchObservedRunningTime="2026-07-14 11:07:14.093213585 +0000 UTC m=+38.675752590"
Jul 14 11:07:14.214427 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:14.214391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:14.351887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:14.351798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:14.986290 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:14.986259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:15.039788 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:15.039757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:15.073842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:15.073813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xkclt" event={"ID":"832fb706-40df-48ff-b0b9-ff498711dc2a","Type":"ContainerStarted","Data":"32cd93939917752fb57797622049b61d47313d0102a77c0b8f28bc053647cbf1"}
Jul 14 11:07:15.088760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:15.088721    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xkclt" podStartSLOduration=1.531043603 podStartE2EDuration="4.088704459s" totalImagesPullingTime="2.557660856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:15.087749739 +0000 UTC m=+39.670288786" watchObservedRunningTime="2026-07-14 11:07:15.088704459 +0000 UTC m=+39.671243465"
Jul 14 11:07:15.144919 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:15.144896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:15.145053 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:15.145033    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:15.145135 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:15.145071    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:15.145188 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:15.145145    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:19.145123607 +0000 UTC m=+43.727662816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:15.245242 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:15.245189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:15.245317 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:15.245304    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:15.245361 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:15.245351    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:07:19.24533947 +0000 UTC m=+43.827878454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:16.270943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:16.270908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:16.368276 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:16.368246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:16.468337 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:16.468308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:16.702497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:16.702462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:17.303608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:17.303567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:17.403907 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:17.403879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:19.171311 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:19.171284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:19.171674 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:19.171388    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:19.171674 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:19.171399    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:19.171674 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:19.171445    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:27.171431994 +0000 UTC m=+51.753970977 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:19.271608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:19.271581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:19.271745 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:19.271711    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:19.271781 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:19.271757    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:07:27.271744571 +0000 UTC m=+51.854283553 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:21.304150 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:21.304107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:21.650694 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:21.650607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:21.751882 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:21.751853    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:24.548216 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:24.548179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:24.609432 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:24.609398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:27.228332 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:27.228295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:27.228729 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:27.228450    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:27.228729 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:27.228468    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:27.228729 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:27.228525    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:43.228507981 +0000 UTC m=+67.811046981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:27.329283 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:27.329249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:27.329436 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:27.329394    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:27.329473 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:27.329456    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:07:43.329441036 +0000 UTC m=+67.911980019 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:28.514599 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:28.514553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:28.569272 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:28.569244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:28.860497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:28.860470    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:28.953417 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:28.953383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:29.053124 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:29.053095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:29.139516 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:29.139444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:32.358383 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:32.358349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:32.418434 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:32.418410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:33.237637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.237603    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"]
Jul 14 11:07:33.269937 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.269912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"]
Jul 14 11:07:33.270062 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.269998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.272441 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.272415    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:07:33.272565 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.272530    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 14 11:07:33.272638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.272625    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2wkh2\""
Jul 14 11:07:33.273361 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.273345    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 14 11:07:33.372495 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.372469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.372833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.372525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brdb8\" (UniqueName: \"kubernetes.io/projected/58a145f1-e14c-43c6-91a8-cd46f9d61cae-kube-api-access-brdb8\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.442536 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.442507    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-qq2tp"]
Jul 14 11:07:33.463882 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.463863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-qq2tp"]
Jul 14 11:07:33.463985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.463950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.466533 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.466512    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 11:07:33.466828 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.466791    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zcb8r\""
Jul 14 11:07:33.466927 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.466910    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 11:07:33.467218 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.467201    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 14 11:07:33.467477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.467461    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 14 11:07:33.475046 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.475017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.475169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.475147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brdb8\" (UniqueName: \"kubernetes.io/projected/58a145f1-e14c-43c6-91a8-cd46f9d61cae-kube-api-access-brdb8\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.475364 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:33.475348    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 11:07:33.475464 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:33.475426    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls podName:58a145f1-e14c-43c6-91a8-cd46f9d61cae nodeName:}" failed. No retries permitted until 2026-07-14 11:07:33.975406554 +0000 UTC m=+58.557945554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls") pod "cluster-samples-operator-65db858894-5v5s5" (UID: "58a145f1-e14c-43c6-91a8-cd46f9d61cae") : secret "samples-operator-tls" not found
Jul 14 11:07:33.476829 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.476806    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 14 11:07:33.502799 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.502744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brdb8\" (UniqueName: \"kubernetes.io/projected/58a145f1-e14c-43c6-91a8-cd46f9d61cae-kube-api-access-brdb8\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.538294 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.538270    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"]
Jul 14 11:07:33.559194 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.559175    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"]
Jul 14 11:07:33.559275 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.559258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.561530 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.561511    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:07:33.561625 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.561549    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-78w5w\""
Jul 14 11:07:33.561739 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.561720    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 14 11:07:33.561819 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.561803    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 14 11:07:33.561924 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.561908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 14 11:07:33.575854 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.575836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5845ce2e-c307-4065-9a4e-78370d482b09-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.575953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.575862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5845ce2e-c307-4065-9a4e-78370d482b09-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.575953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.575878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3da78d2b-82fb-4bcf-a7c7-518c8387914a-tmp\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.575953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.575902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da78d2b-82fb-4bcf-a7c7-518c8387914a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.575953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.575937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hq7s\" (UniqueName: \"kubernetes.io/projected/3da78d2b-82fb-4bcf-a7c7-518c8387914a-kube-api-access-5hq7s\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.576162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.576002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da78d2b-82fb-4bcf-a7c7-518c8387914a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.576162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.576082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8m6\" (UniqueName: \"kubernetes.io/projected/5845ce2e-c307-4065-9a4e-78370d482b09-kube-api-access-bb8m6\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.576162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.576121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3da78d2b-82fb-4bcf-a7c7-518c8387914a-serving-cert\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.576262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.576178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3da78d2b-82fb-4bcf-a7c7-518c8387914a-snapshots\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.592265 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.592238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:33.653184 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.653143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:33.677317 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da78d2b-82fb-4bcf-a7c7-518c8387914a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.677388 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8m6\" (UniqueName: \"kubernetes.io/projected/5845ce2e-c307-4065-9a4e-78370d482b09-kube-api-access-bb8m6\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.677388 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3da78d2b-82fb-4bcf-a7c7-518c8387914a-serving-cert\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.677464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3da78d2b-82fb-4bcf-a7c7-518c8387914a-snapshots\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.677576 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5845ce2e-c307-4065-9a4e-78370d482b09-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.677635 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5845ce2e-c307-4065-9a4e-78370d482b09-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.677635 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3da78d2b-82fb-4bcf-a7c7-518c8387914a-tmp\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.677765 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da78d2b-82fb-4bcf-a7c7-518c8387914a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.677765 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hq7s\" (UniqueName: \"kubernetes.io/projected/3da78d2b-82fb-4bcf-a7c7-518c8387914a-kube-api-access-5hq7s\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.677908 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da78d2b-82fb-4bcf-a7c7-518c8387914a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.678030 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.677954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3da78d2b-82fb-4bcf-a7c7-518c8387914a-tmp\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.678124 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.678101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3da78d2b-82fb-4bcf-a7c7-518c8387914a-snapshots\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.678320 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.678300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5845ce2e-c307-4065-9a4e-78370d482b09-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.678510 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.678488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da78d2b-82fb-4bcf-a7c7-518c8387914a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.679638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.679621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3da78d2b-82fb-4bcf-a7c7-518c8387914a-serving-cert\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.679730 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.679706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5845ce2e-c307-4065-9a4e-78370d482b09-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.684648 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.684623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8m6\" (UniqueName: \"kubernetes.io/projected/5845ce2e-c307-4065-9a4e-78370d482b09-kube-api-access-bb8m6\") pod \"kube-storage-version-migrator-operator-799cb7f958-cnggk\" (UID: \"5845ce2e-c307-4065-9a4e-78370d482b09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.684745 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.684622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hq7s\" (UniqueName: \"kubernetes.io/projected/3da78d2b-82fb-4bcf-a7c7-518c8387914a-kube-api-access-5hq7s\") pod \"insights-operator-68d9dd94cf-qq2tp\" (UID: \"3da78d2b-82fb-4bcf-a7c7-518c8387914a\") " pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.772957 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.772937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp"
Jul 14 11:07:33.867996 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.867968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"
Jul 14 11:07:33.919109 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.919078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-qq2tp"]
Jul 14 11:07:33.922826 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:07:33.922798    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da78d2b_82fb_4bcf_a7c7_518c8387914a.slice/crio-6fee9fd2c98cfbbfcc7d76cc9b917156c24a4f3d7f167fc072607e1ddbb9a0da WatchSource:0}: Error finding container 6fee9fd2c98cfbbfcc7d76cc9b917156c24a4f3d7f167fc072607e1ddbb9a0da: Status 404 returned error can't find the container with id 6fee9fd2c98cfbbfcc7d76cc9b917156c24a4f3d7f167fc072607e1ddbb9a0da
Jul 14 11:07:33.980218 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:33.980184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:33.980356 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:33.980335    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 11:07:33.980412 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:33.980403    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls podName:58a145f1-e14c-43c6-91a8-cd46f9d61cae nodeName:}" failed. No retries permitted until 2026-07-14 11:07:34.980386913 +0000 UTC m=+59.562925895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls") pod "cluster-samples-operator-65db858894-5v5s5" (UID: "58a145f1-e14c-43c6-91a8-cd46f9d61cae") : secret "samples-operator-tls" not found
Jul 14 11:07:34.006717 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:34.006695    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk"]
Jul 14 11:07:34.009567 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:07:34.009540    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5845ce2e_c307_4065_9a4e_78370d482b09.slice/crio-f222fffbf38c034f00baa3a69b453b251601091adc388873022406384d729e79 WatchSource:0}: Error finding container f222fffbf38c034f00baa3a69b453b251601091adc388873022406384d729e79: Status 404 returned error can't find the container with id f222fffbf38c034f00baa3a69b453b251601091adc388873022406384d729e79
Jul 14 11:07:34.117851 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:34.117768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk" event={"ID":"5845ce2e-c307-4065-9a4e-78370d482b09","Type":"ContainerStarted","Data":"f222fffbf38c034f00baa3a69b453b251601091adc388873022406384d729e79"}
Jul 14 11:07:34.118898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:34.118874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp" event={"ID":"3da78d2b-82fb-4bcf-a7c7-518c8387914a","Type":"ContainerStarted","Data":"6fee9fd2c98cfbbfcc7d76cc9b917156c24a4f3d7f167fc072607e1ddbb9a0da"}
Jul 14 11:07:34.988861 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:34.988819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:34.989308 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:34.989015    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 11:07:34.989308 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:34.989098    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls podName:58a145f1-e14c-43c6-91a8-cd46f9d61cae nodeName:}" failed. No retries permitted until 2026-07-14 11:07:36.989075848 +0000 UTC m=+61.571614848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls") pod "cluster-samples-operator-65db858894-5v5s5" (UID: "58a145f1-e14c-43c6-91a8-cd46f9d61cae") : secret "samples-operator-tls" not found
Jul 14 11:07:35.048669 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:35.048625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5675s"
Jul 14 11:07:35.053536 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:35.053511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:35.154167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:35.154138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:35.257445 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:35.257363    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:35.358122 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:35.358089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:36.125930 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:36.125839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp" event={"ID":"3da78d2b-82fb-4bcf-a7c7-518c8387914a","Type":"ContainerStarted","Data":"9b8dfd6123d418546dfa1011eeb434f633f54b4c630ea18e064085d9b2af27b8"}
Jul 14 11:07:36.142060 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:36.142006    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp" podStartSLOduration=1.19003575 podStartE2EDuration="3.141987917s" totalImagesPullingTime="1.951952167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:36.14118067 +0000 UTC m=+60.723719676" watchObservedRunningTime="2026-07-14 11:07:36.141987917 +0000 UTC m=+60.724526925"
Jul 14 11:07:36.350507 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:36.350464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:36.404509 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:36.404405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:37.008132 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.008098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:37.008296 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:37.008264    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 11:07:37.008348 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:37.008337    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls podName:58a145f1-e14c-43c6-91a8-cd46f9d61cae nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.008318254 +0000 UTC m=+65.590857240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls") pod "cluster-samples-operator-65db858894-5v5s5" (UID: "58a145f1-e14c-43c6-91a8-cd46f9d61cae") : secret "samples-operator-tls" not found
Jul 14 11:07:37.130158 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.130123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk" event={"ID":"5845ce2e-c307-4065-9a4e-78370d482b09","Type":"ContainerStarted","Data":"5897801a03f46185770d9b1e07acc7941f2bf1b43680fdb6a01622ba41860f59"}
Jul 14 11:07:37.148665 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.148610    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk" podStartSLOduration=1.586884593 podStartE2EDuration="4.148590882s" totalImagesPullingTime="2.561706289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:37.147870007 +0000 UTC m=+61.730409012" watchObservedRunningTime="2026-07-14 11:07:37.148590882 +0000 UTC m=+61.731129892"
Jul 14 11:07:37.426076 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.426045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zrhmb"]
Jul 14 11:07:37.428059 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.428036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.431898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.430903    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 11:07:37.431898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.431126    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 11:07:37.431898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.431296    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-24zbp\""
Jul 14 11:07:37.456258 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.456238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zrhmb"]
Jul 14 11:07:37.461689 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.461644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:37.512955 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.512924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/33cb34de-9808-4cc6-9f33-9d2006af17d4-crio-socket\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.513071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.512975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbprp\" (UniqueName: \"kubernetes.io/projected/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-api-access-cbprp\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.513071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.513000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.513071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.513051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/33cb34de-9808-4cc6-9f33-9d2006af17d4-data-volume\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.513168 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.513130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.521628 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.521607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:37.596408 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.596374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:37.614439 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/33cb34de-9808-4cc6-9f33-9d2006af17d4-crio-socket\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.614577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbprp\" (UniqueName: \"kubernetes.io/projected/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-api-access-cbprp\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.614577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.614577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/33cb34de-9808-4cc6-9f33-9d2006af17d4-data-volume\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.614577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/33cb34de-9808-4cc6-9f33-9d2006af17d4-crio-socket\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.614973 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:37.614578    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:07:37.614973 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:37.614620    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:37.614973 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:37.614643    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:38.114626615 +0000 UTC m=+62.697165598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : configmap "kube-rbac-proxy" not found
Jul 14 11:07:37.614973 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.614973 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:37.614782    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:38.114768026 +0000 UTC m=+62.697307026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:37.614973 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.614964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/33cb34de-9808-4cc6-9f33-9d2006af17d4-data-volume\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.626970 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.626949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbprp\" (UniqueName: \"kubernetes.io/projected/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-api-access-cbprp\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:37.697328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:37.697265    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:38.117919 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:38.117891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:38.118150 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:38.118043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:38.118150 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:38.118045    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:38.118150 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:38.118110    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:07:38.118150 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:38.118119    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:39.118104005 +0000 UTC m=+63.700642991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:38.118286 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:38.118160    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:39.118148688 +0000 UTC m=+63.700687671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : configmap "kube-rbac-proxy" not found
Jul 14 11:07:38.698563 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:38.698529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:38.810935 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:38.810905    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:39.125699 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:39.125669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:39.125848 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:39.125789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:39.125848 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:39.125800    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:39.125922 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:39.125861    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.125843258 +0000 UTC m=+65.708382264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:39.125922 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:39.125907    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:07:39.126018 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:39.126006    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.125990784 +0000 UTC m=+65.708529774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : configmap "kube-rbac-proxy" not found
Jul 14 11:07:39.471783 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:39.471708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q65sx_0d5a1d51-4bae-4c57-87f8-46ccd09e8d52/dns-node-resolver/0.log"
Jul 14 11:07:39.832830 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:39.832804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:39.922882 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:39.922854    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:40.203882 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:40.203801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:40.306470 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:40.306432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:40.469695 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:40.469615    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zxxqq_2abf4d3e-c2ff-458c-beb2-f45063b7a331/node-ca/0.log"
Jul 14 11:07:40.478867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:40.478840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:40.578455 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:40.578428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:41.040127 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.040096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:41.040468 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.040215    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 11:07:41.040468 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.040278    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls podName:58a145f1-e14c-43c6-91a8-cd46f9d61cae nodeName:}" failed. No retries permitted until 2026-07-14 11:07:49.040262107 +0000 UTC m=+73.622801091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls") pod "cluster-samples-operator-65db858894-5v5s5" (UID: "58a145f1-e14c-43c6-91a8-cd46f9d61cae") : secret "samples-operator-tls" not found
Jul 14 11:07:41.067802 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.067772    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xkclt_832fb706-40df-48ff-b0b9-ff498711dc2a/serve-healthcheck-canary/0.log"
Jul 14 11:07:41.140997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.140970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:41.141160 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.141023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:41.141160 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.141095    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:07:41.141160 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.141153    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:41.141260 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.141163    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:45.141146 +0000 UTC m=+69.723684985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : configmap "kube-rbac-proxy" not found
Jul 14 11:07:41.141260 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.141189    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:45.141179095 +0000 UTC m=+69.723718079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:41.545199 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.545169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:41.545366 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.545264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:07:41.547720 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.547702    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 11:07:41.547810 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.547705    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 11:07:41.555683 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.555647    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 11:07:41.555760 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:41.555721    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs podName:558289b9-aa16-4008-a8fd-4eaa7271e009 nodeName:}" failed. No retries permitted until 2026-07-14 11:08:45.555701105 +0000 UTC m=+130.138240105 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs") pod "network-metrics-daemon-xlw8z" (UID: "558289b9-aa16-4008-a8fd-4eaa7271e009") : secret "metrics-daemon-secret" not found
Jul 14 11:07:41.557739 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.557723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3e6c755-ee41-48b9-81d4-7caf68a60441-original-pull-secret\") pod \"global-pull-secret-syncer-tb8l9\" (UID: \"c3e6c755-ee41-48b9-81d4-7caf68a60441\") " pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:41.746962 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.746920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:41.749414 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.749399    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 11:07:41.760365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.760343    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 11:07:41.770480 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.770457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7wsz\" (UniqueName: \"kubernetes.io/projected/28f7113a-277a-4ace-9914-c288126f7bd9-kube-api-access-k7wsz\") pod \"network-check-target-k24x7\" (UID: \"28f7113a-277a-4ace-9914-c288126f7bd9\") " pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:41.833456 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.833411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tb8l9"
Jul 14 11:07:41.935673 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.935636    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zjjh4\""
Jul 14 11:07:41.943915 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.943891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:41.948238 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:41.948218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tb8l9"]
Jul 14 11:07:41.952222 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:07:41.952204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e6c755_ee41_48b9_81d4_7caf68a60441.slice/crio-c691157aa0094d61241eded1a89b25db12d88c0bc0aedf68465b5ce1f54ba225 WatchSource:0}: Error finding container c691157aa0094d61241eded1a89b25db12d88c0bc0aedf68465b5ce1f54ba225: Status 404 returned error can't find the container with id c691157aa0094d61241eded1a89b25db12d88c0bc0aedf68465b5ce1f54ba225
Jul 14 11:07:42.063154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.063111    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8mdg8"]
Jul 14 11:07:42.068466 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.068441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.069817 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.069795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k24x7"]
Jul 14 11:07:42.071858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.071837    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 14 11:07:42.071858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.071847    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 14 11:07:42.072022 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.071875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 14 11:07:42.072022 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.072011    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dqhl7\""
Jul 14 11:07:42.072142 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.072124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 14 11:07:42.073553 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:07:42.073535    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f7113a_277a_4ace_9914_c288126f7bd9.slice/crio-10a54789862711b82c3d227edd54d97cd65de7584cdf4f0a337ef011af500617 WatchSource:0}: Error finding container 10a54789862711b82c3d227edd54d97cd65de7584cdf4f0a337ef011af500617: Status 404 returned error can't find the container with id 10a54789862711b82c3d227edd54d97cd65de7584cdf4f0a337ef011af500617
Jul 14 11:07:42.076538 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.076446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8mdg8"]
Jul 14 11:07:42.098410 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.098347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:42.142509 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.142488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k24x7" event={"ID":"28f7113a-277a-4ace-9914-c288126f7bd9","Type":"ContainerStarted","Data":"10a54789862711b82c3d227edd54d97cd65de7584cdf4f0a337ef011af500617"}
Jul 14 11:07:42.143467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.143449    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tb8l9" event={"ID":"c3e6c755-ee41-48b9-81d4-7caf68a60441","Type":"ContainerStarted","Data":"c691157aa0094d61241eded1a89b25db12d88c0bc0aedf68465b5ce1f54ba225"}
Jul 14 11:07:42.150813 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.150794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xj6k\" (UniqueName: \"kubernetes.io/projected/08c282cc-fbe8-4b96-862b-6cab61e1046a-kube-api-access-8xj6k\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.150895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.150833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08c282cc-fbe8-4b96-862b-6cab61e1046a-signing-key\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.150963 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.150949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08c282cc-fbe8-4b96-862b-6cab61e1046a-signing-cabundle\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.251535 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.251508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xj6k\" (UniqueName: \"kubernetes.io/projected/08c282cc-fbe8-4b96-862b-6cab61e1046a-kube-api-access-8xj6k\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.251630 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.251543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08c282cc-fbe8-4b96-862b-6cab61e1046a-signing-key\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.251630 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.251613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08c282cc-fbe8-4b96-862b-6cab61e1046a-signing-cabundle\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.252181 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.252158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08c282cc-fbe8-4b96-862b-6cab61e1046a-signing-cabundle\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.253928 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.253912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08c282cc-fbe8-4b96-862b-6cab61e1046a-signing-key\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.259131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.259114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xj6k\" (UniqueName: \"kubernetes.io/projected/08c282cc-fbe8-4b96-862b-6cab61e1046a-kube-api-access-8xj6k\") pod \"service-ca-79f68b9f4f-8mdg8\" (UID: \"08c282cc-fbe8-4b96-862b-6cab61e1046a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.379760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.379650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8"
Jul 14 11:07:42.384418 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.384391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:42.442566 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.442541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:42.499116 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:42.499081    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8mdg8"]
Jul 14 11:07:42.502417 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:07:42.502391    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c282cc_fbe8_4b96_862b_6cab61e1046a.slice/crio-4cea8c4607bafea666094c58bd9900b243dc125e6766c4743ec72e015abd7cf6 WatchSource:0}: Error finding container 4cea8c4607bafea666094c58bd9900b243dc125e6766c4743ec72e015abd7cf6: Status 404 returned error can't find the container with id 4cea8c4607bafea666094c58bd9900b243dc125e6766c4743ec72e015abd7cf6
Jul 14 11:07:43.040496 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:43.040439    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:43.148593 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:43.148558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8" event={"ID":"08c282cc-fbe8-4b96-862b-6cab61e1046a","Type":"ContainerStarted","Data":"4cea8c4607bafea666094c58bd9900b243dc125e6766c4743ec72e015abd7cf6"}
Jul 14 11:07:43.261228 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:43.261190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:07:43.261402 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:43.261362    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 11:07:43.261402 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:43.261385    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657c88457b-mrjlh: secret "image-registry-tls" not found
Jul 14 11:07:43.261511 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:43.261459    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls podName:1c9fd828-d26d-4dca-b75a-16f200bec483 nodeName:}" failed. No retries permitted until 2026-07-14 11:08:15.261437844 +0000 UTC m=+99.843976849 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls") pod "image-registry-657c88457b-mrjlh" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483") : secret "image-registry-tls" not found
Jul 14 11:07:43.362538 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:43.362409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:07:43.362765 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:43.362571    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:07:43.362765 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:43.362649    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls podName:247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c nodeName:}" failed. No retries permitted until 2026-07-14 11:08:15.362629405 +0000 UTC m=+99.945168405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls") pod "dns-default-htvd2" (UID: "247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c") : secret "dns-default-metrics-tls" not found
Jul 14 11:07:45.180114 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:45.180086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:45.180521 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:45.180145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:45.180521 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:45.180232    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:07:45.180521 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:45.180267    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:45.180521 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:45.180293    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:53.180278305 +0000 UTC m=+77.762817292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : configmap "kube-rbac-proxy" not found
Jul 14 11:07:45.180521 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:45.180308    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:53.180301292 +0000 UTC m=+77.762840278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:46.692217 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:46.692153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:48.166644 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.166609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tb8l9" event={"ID":"c3e6c755-ee41-48b9-81d4-7caf68a60441","Type":"ContainerStarted","Data":"12eaf8355744cec6498c8836df41e224083a1b31eaa2eed7b32cf3498eee4dfc"}
Jul 14 11:07:48.168167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.168136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8" event={"ID":"08c282cc-fbe8-4b96-862b-6cab61e1046a","Type":"ContainerStarted","Data":"aa825890388c966a2a2de4320f297e8361918287c36402ba335d96f279632c5b"}
Jul 14 11:07:48.169480 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.169457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k24x7" event={"ID":"28f7113a-277a-4ace-9914-c288126f7bd9","Type":"ContainerStarted","Data":"7ecee1ba3bb6e1df6bf7297494794fb7a1e48bf29ab453b39e0999f3fbf4ae6d"}
Jul 14 11:07:48.169603 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.169588    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:07:48.184895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.184841    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tb8l9" podStartSLOduration=66.499169046 podStartE2EDuration="1m12.184826476s" totalImagesPullingTime="5.68565743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:48.183813137 +0000 UTC m=+72.766352139" watchObservedRunningTime="2026-07-14 11:07:48.184826476 +0000 UTC m=+72.767365481"
Jul 14 11:07:48.200866 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.200826    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-8mdg8" podStartSLOduration=1.075810501 podStartE2EDuration="6.200814384s" totalImagesPullingTime="5.125003883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:48.199962627 +0000 UTC m=+72.782501631" watchObservedRunningTime="2026-07-14 11:07:48.200814384 +0000 UTC m=+72.783353389"
Jul 14 11:07:48.219094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.219041    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k24x7" podStartSLOduration=66.666223208 podStartE2EDuration="1m12.219023791s" totalImagesPullingTime="5.552800583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:48.217323167 +0000 UTC m=+72.799862171" watchObservedRunningTime="2026-07-14 11:07:48.219023791 +0000 UTC m=+72.801562801"
Jul 14 11:07:48.556964 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.556870    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:48.704047 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.704010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:48.801616 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.801577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:48.981049 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:48.980970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:49.080715 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:49.080687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:49.113700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:49.113670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:07:49.113851 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:49.113831    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 11:07:49.113914 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:49.113903    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls podName:58a145f1-e14c-43c6-91a8-cd46f9d61cae nodeName:}" failed. No retries permitted until 2026-07-14 11:08:05.113885165 +0000 UTC m=+89.696424150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls") pod "cluster-samples-operator-65db858894-5v5s5" (UID: "58a145f1-e14c-43c6-91a8-cd46f9d61cae") : secret "samples-operator-tls" not found
Jul 14 11:07:49.264348 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:49.264268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:50.053602 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:50.053566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:50.476913 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:50.476813    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:50.629997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:50.629965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:50.868163 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:50.868136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:51.053791 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:51.053735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:51.269356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:51.269273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:51.649125 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:51.649090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:51.705176 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:51.705150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:52.051686 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:52.051643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:52.251667 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:52.251621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:52.449813 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:52.449729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:52.548229 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:52.548203    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:52.847574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:52.847543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:53.050986 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:53.050948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:53.108826 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:53.108747    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:53.249039 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:53.249015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:53.249188 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:53.249098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:53.249188 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:53.249175    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:53.249276 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:07:53.249246    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls podName:33cb34de-9808-4cc6-9f33-9d2006af17d4 nodeName:}" failed. No retries permitted until 2026-07-14 11:08:09.249225286 +0000 UTC m=+93.831764283 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zrhmb" (UID: "33cb34de-9808-4cc6-9f33-9d2006af17d4") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:07:53.249515 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:53.249498    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/33cb34de-9808-4cc6-9f33-9d2006af17d4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:07:53.484515 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:53.484422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:54.051616 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:54.051583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:54.286488 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:54.286448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:54.451246 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:54.451168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:54.859637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:54.859601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:54.960479 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:54.960442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:55.244956 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:55.244879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:55.458018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:55.457984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:56.045466 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:56.045434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:56.455871 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:56.455773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:56.647916 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:56.647879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:57.245642 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:57.245605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:57.851159 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:57.851037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.048338 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.048305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.259495 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.259409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.439843 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.439813    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.541385 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.541331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.649227 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.649202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.712506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.712478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:58.910451 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:58.910386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:59.063839 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:59.063813    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:59.132564 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:59.132541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:59.250942 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:59.250872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:07:59.351368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:07:59.351343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:00.284086 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:00.284054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:00.475806 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:00.475774    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:00.787690 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:00.787635    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:04.855090 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.855057    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"]
Jul 14 11:08:04.859483 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.859462    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:04.862167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.862150    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 14 11:08:04.862417 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.862405    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 14 11:08:04.862470 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.862412    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 14 11:08:04.863020 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.863007    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 14 11:08:04.863066 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.863009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 14 11:08:04.863288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.863273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 14 11:08:04.863396 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.863383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 14 11:08:04.881821 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.881802    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"]
Jul 14 11:08:04.988570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:04.988538    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:05.035639 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.035611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vgx6\" (UniqueName: \"kubernetes.io/projected/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-kube-api-access-6vgx6\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.035639 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.035640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.035827 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.035724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-hub\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.035827 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.035784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-ca\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.035827 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.035800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.035827 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.035821    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.084157 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.084124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:05.136201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-ca\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.136201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.136201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.136391 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vgx6\" (UniqueName: \"kubernetes.io/projected/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-kube-api-access-6vgx6\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.136391 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.136485 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-hub\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.136485 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.136448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:08:05.137029 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.137002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.138594 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.138565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-ca\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.138858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.138838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.138929 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.138911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.139140 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.139111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-hub\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.139271 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.139253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/58a145f1-e14c-43c6-91a8-cd46f9d61cae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-5v5s5\" (UID: \"58a145f1-e14c-43c6-91a8-cd46f9d61cae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:08:05.145064 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.145040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vgx6\" (UniqueName: \"kubernetes.io/projected/6141ec7c-ae04-4a15-9f4f-2924a7d8fe05-kube-api-access-6vgx6\") pod \"cluster-proxy-proxy-agent-7674f5889b-6hh5f\" (UID: \"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.178553 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.178527    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"
Jul 14 11:08:05.184525 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.184499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:05.277921 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.277886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:05.318462 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.318433    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f"]
Jul 14 11:08:05.327511 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:08:05.327482    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6141ec7c_ae04_4a15_9f4f_2924a7d8fe05.slice/crio-8b3daf31799571bad26bf764c332436d620b2e05ba24ff6a663d1de7f7068fb7 WatchSource:0}: Error finding container 8b3daf31799571bad26bf764c332436d620b2e05ba24ff6a663d1de7f7068fb7: Status 404 returned error can't find the container with id 8b3daf31799571bad26bf764c332436d620b2e05ba24ff6a663d1de7f7068fb7
Jul 14 11:08:05.381393 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.381363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2wkh2\""
Jul 14 11:08:05.389938 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.389871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"
Jul 14 11:08:05.423307 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.423268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:05.515825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.515777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5"]
Jul 14 11:08:05.523869 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:05.523846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:06.224188 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:06.224152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5" event={"ID":"58a145f1-e14c-43c6-91a8-cd46f9d61cae","Type":"ContainerStarted","Data":"20a4baa64444831423b509568ee62c058cbb281add9a3840c64c3695df1f307c"}
Jul 14 11:08:06.225435 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:06.225411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f" event={"ID":"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05","Type":"ContainerStarted","Data":"8b3daf31799571bad26bf764c332436d620b2e05ba24ff6a663d1de7f7068fb7"}
Jul 14 11:08:06.972341 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:06.972283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:07.052637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:07.052582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:08.682969 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:08.682903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:08.784107 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:08.784072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:08.859947 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:08.859916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:09.271168 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:09.271130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:08:09.273621 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:09.273601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/33cb34de-9808-4cc6-9f33-9d2006af17d4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zrhmb\" (UID: \"33cb34de-9808-4cc6-9f33-9d2006af17d4\") " pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:08:09.437094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:09.437053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:09.541045 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:09.541021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zrhmb"
Jul 14 11:08:09.824222 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:09.824205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zrhmb"]
Jul 14 11:08:09.825520 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:08:09.825497    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33cb34de_9808_4cc6_9f33_9d2006af17d4.slice/crio-e750ac010ca508fbd12ff32efb33b0ff8614d3b66530493987199e62f91fc225 WatchSource:0}: Error finding container e750ac010ca508fbd12ff32efb33b0ff8614d3b66530493987199e62f91fc225: Status 404 returned error can't find the container with id e750ac010ca508fbd12ff32efb33b0ff8614d3b66530493987199e62f91fc225
Jul 14 11:08:10.241043 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.241008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5" event={"ID":"58a145f1-e14c-43c6-91a8-cd46f9d61cae","Type":"ContainerStarted","Data":"ce4fdbfefc2aaf58d22f4a7a1f13070683ec6d44090357af8dd79e2993d5a65e"}
Jul 14 11:08:10.241287 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.241061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5" event={"ID":"58a145f1-e14c-43c6-91a8-cd46f9d61cae","Type":"ContainerStarted","Data":"efbe9f7f0bea004c10d255139a2ae7a3292ad3fed01b5652b5e0a65d64cddf58"}
Jul 14 11:08:10.242178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.242156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zrhmb" event={"ID":"33cb34de-9808-4cc6-9f33-9d2006af17d4","Type":"ContainerStarted","Data":"e750ac010ca508fbd12ff32efb33b0ff8614d3b66530493987199e62f91fc225"}
Jul 14 11:08:10.244002 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.243974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f" event={"ID":"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05","Type":"ContainerStarted","Data":"6b4f4d42cc818cb990b35e5fe3ee841c35b666bdb6622c8e2d6e4fc083bcf349"}
Jul 14 11:08:10.244070 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.244007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f" event={"ID":"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05","Type":"ContainerStarted","Data":"714d7c13a649a7e38af2427901c4136aaa974cc630dabf560ba03437a9087589"}
Jul 14 11:08:10.244070 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.244019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f" event={"ID":"6141ec7c-ae04-4a15-9f4f-2924a7d8fe05","Type":"ContainerStarted","Data":"36ecd2c9caa34fe515c0bd8faead0be8ed975281c49dd785a6059006071fdbc4"}
Jul 14 11:08:10.265618 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.265567    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-5v5s5" podStartSLOduration=33.31184468 podStartE2EDuration="37.265553891s" totalImagesPullingTime="3.953709211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:10.265445058 +0000 UTC m=+94.847984095" watchObservedRunningTime="2026-07-14 11:08:10.265553891 +0000 UTC m=+94.848092896"
Jul 14 11:08:10.303458 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.303407    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7674f5889b-6hh5f" podStartSLOduration=1.917292959 podStartE2EDuration="6.303389127s" totalImagesPullingTime="4.386096168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:10.302831888 +0000 UTC m=+94.885370929" watchObservedRunningTime="2026-07-14 11:08:10.303389127 +0000 UTC m=+94.885928135"
Jul 14 11:08:10.914668 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.914613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:10.974055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:10.974013    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:11.877678 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:11.877547    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:11.956625 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:11.956580    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:12.014610 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:12.014569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:14.260348 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:14.260316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zrhmb" event={"ID":"33cb34de-9808-4cc6-9f33-9d2006af17d4","Type":"ContainerStarted","Data":"c8b237abba000df9a0ae8571c0a857b9330664ccb70eaaffd3cd16e90f226a92"}
Jul 14 11:08:14.260626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:14.260372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zrhmb" event={"ID":"33cb34de-9808-4cc6-9f33-9d2006af17d4","Type":"ContainerStarted","Data":"1e3bba195e6e39cec73daf59df46d1e4f50179796f91474c85d797b2d20b5828"}
Jul 14 11:08:14.301401 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:14.301375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:14.358207 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:14.358182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:15.323966 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.323931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:15.326694 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.326648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"image-registry-657c88457b-mrjlh\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") " pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:15.425091 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.425058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:08:15.427470 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.427443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c-metrics-tls\") pod \"dns-default-htvd2\" (UID: \"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c\") " pod="openshift-dns/dns-default-htvd2"
Jul 14 11:08:15.596203 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.596128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cn4jk\""
Jul 14 11:08:15.604236 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.604168    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:15.635277 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.635251    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-grcgd\""
Jul 14 11:08:15.643387 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.643365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-htvd2"
Jul 14 11:08:15.952435 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:15.952360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:16.709483 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:16.709445    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:16.776453 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:16.776420    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:16.858550 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:16.858513    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-htvd2"]
Jul 14 11:08:16.877597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:16.877575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-657c88457b-mrjlh"]
Jul 14 11:08:17.065955 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:08:17.065915    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247fa4f5_0c27_4aa7_ad3f_87d2cc0bc09c.slice/crio-08fbaa7a473aac0c59ebaa8919d819d501dfe64a58572847cb00d6ab7c3c429d WatchSource:0}: Error finding container 08fbaa7a473aac0c59ebaa8919d819d501dfe64a58572847cb00d6ab7c3c429d: Status 404 returned error can't find the container with id 08fbaa7a473aac0c59ebaa8919d819d501dfe64a58572847cb00d6ab7c3c429d
Jul 14 11:08:17.066639 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:08:17.066518    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9fd828_d26d_4dca_b75a_16f200bec483.slice/crio-d5a16d56567bf51b8d3c9ef7be5883246c24b732bd527d32dfd8e594776009e2 WatchSource:0}: Error finding container d5a16d56567bf51b8d3c9ef7be5883246c24b732bd527d32dfd8e594776009e2: Status 404 returned error can't find the container with id d5a16d56567bf51b8d3c9ef7be5883246c24b732bd527d32dfd8e594776009e2
Jul 14 11:08:17.272318 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.272280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zrhmb" event={"ID":"33cb34de-9808-4cc6-9f33-9d2006af17d4","Type":"ContainerStarted","Data":"392da1dfb379e33aef6f885dea16ef4569ea83277d0663a51105acfa93c572f8"}
Jul 14 11:08:17.273418 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.273396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-htvd2" event={"ID":"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c","Type":"ContainerStarted","Data":"08fbaa7a473aac0c59ebaa8919d819d501dfe64a58572847cb00d6ab7c3c429d"}
Jul 14 11:08:17.274704 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.274686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657c88457b-mrjlh" event={"ID":"1c9fd828-d26d-4dca-b75a-16f200bec483","Type":"ContainerStarted","Data":"377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b"}
Jul 14 11:08:17.274801 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.274716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657c88457b-mrjlh" event={"ID":"1c9fd828-d26d-4dca-b75a-16f200bec483","Type":"ContainerStarted","Data":"d5a16d56567bf51b8d3c9ef7be5883246c24b732bd527d32dfd8e594776009e2"}
Jul 14 11:08:17.274870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.274820    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:17.291932 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.291889    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zrhmb" podStartSLOduration=33.122827343 podStartE2EDuration="40.291871484s" totalImagesPullingTime="7.169044141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:07:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:17.291166942 +0000 UTC m=+101.873705941" watchObservedRunningTime="2026-07-14 11:08:17.291871484 +0000 UTC m=+101.874410488"
Jul 14 11:08:17.313034 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.312987    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-657c88457b-mrjlh" podStartSLOduration=101.312972289 podStartE2EDuration="1m41.312972289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:17.312027962 +0000 UTC m=+101.894566978" watchObservedRunningTime="2026-07-14 11:08:17.312972289 +0000 UTC m=+101.895511293"
Jul 14 11:08:17.523281 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.523236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:17.586261 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:17.586228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.050852 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.050630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.246696 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.246613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.312052 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.311966    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vpx7j"]
Jul 14 11:08:18.328016 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.327983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.330227 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.330203    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 11:08:18.330356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.330279    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 11:08:18.330514 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.330498    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 11:08:18.330565 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.330518    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 11:08:18.331171 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.331140    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7hgj\""
Jul 14 11:08:18.331288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.331245    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 11:08:18.331288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.331265    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 11:08:18.345190 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e415343-0445-463e-9139-b8b47765a898-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.345284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-root\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.345284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-node-exporter-wtmp\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.345284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e415343-0445-463e-9139-b8b47765a898-metrics-client-ca\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.345442 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tcrl\" (UniqueName: \"kubernetes.io/projected/7e415343-0445-463e-9139-b8b47765a898-kube-api-access-6tcrl\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.345442 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345351    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.345925 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-sys\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.346060 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.345934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e415343-0445-463e-9139-b8b47765a898-node-exporter-accelerators-collector-config\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.346128 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.346061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e415343-0445-463e-9139-b8b47765a898-node-exporter-textfile\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.346183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.346137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e415343-0445-463e-9139-b8b47765a898-node-exporter-tls\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.445150 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.445104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.446899 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.446871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tcrl\" (UniqueName: \"kubernetes.io/projected/7e415343-0445-463e-9139-b8b47765a898-kube-api-access-6tcrl\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.446968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-sys\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.446996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e415343-0445-463e-9139-b8b47765a898-node-exporter-accelerators-collector-config\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e415343-0445-463e-9139-b8b47765a898-node-exporter-textfile\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447170 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e415343-0445-463e-9139-b8b47765a898-node-exporter-tls\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447170 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e415343-0445-463e-9139-b8b47765a898-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447170 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-root\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447170 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-node-exporter-wtmp\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447357 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e415343-0445-463e-9139-b8b47765a898-metrics-client-ca\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.447494 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-sys\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.448014 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-root\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.448014 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e415343-0445-463e-9139-b8b47765a898-node-exporter-textfile\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.448014 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e415343-0445-463e-9139-b8b47765a898-node-exporter-wtmp\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.448014 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e415343-0445-463e-9139-b8b47765a898-metrics-client-ca\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.448014 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.447962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e415343-0445-463e-9139-b8b47765a898-node-exporter-accelerators-collector-config\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.449643 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.449622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e415343-0445-463e-9139-b8b47765a898-node-exporter-tls\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.449745 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.449625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e415343-0445-463e-9139-b8b47765a898-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.454729 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.454701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tcrl\" (UniqueName: \"kubernetes.io/projected/7e415343-0445-463e-9139-b8b47765a898-kube-api-access-6tcrl\") pod \"node-exporter-vpx7j\" (UID: \"7e415343-0445-463e-9139-b8b47765a898\") " pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.547081 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.547031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.640886 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.640811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vpx7j"
Jul 14 11:08:18.651020 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.650986    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.744473 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.744440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:18.851019 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:18.850971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:19.050803 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:08:19.050777    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e415343_0445_463e_9139_b8b47765a898.slice/crio-834d473d9ed06fa1643ccd1fe4b92a050fcb540e0310f2de4041f9de5404a1dc WatchSource:0}: Error finding container 834d473d9ed06fa1643ccd1fe4b92a050fcb540e0310f2de4041f9de5404a1dc: Status 404 returned error can't find the container with id 834d473d9ed06fa1643ccd1fe4b92a050fcb540e0310f2de4041f9de5404a1dc
Jul 14 11:08:19.175794 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:19.175768    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k24x7"
Jul 14 11:08:19.263419 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:19.263385    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:19.282841 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:19.282809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vpx7j" event={"ID":"7e415343-0445-463e-9139-b8b47765a898","Type":"ContainerStarted","Data":"834d473d9ed06fa1643ccd1fe4b92a050fcb540e0310f2de4041f9de5404a1dc"}
Jul 14 11:08:19.353887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:19.353780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:19.487579 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:19.487551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:19.959305 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:19.959222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:20.288018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:20.287982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-htvd2" event={"ID":"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c","Type":"ContainerStarted","Data":"19b1f0eb0cce093743f4b6e0d49c56234e074806209ee502ad8935b937a47732"}
Jul 14 11:08:20.288454 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:20.288046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-htvd2" event={"ID":"247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c","Type":"ContainerStarted","Data":"ea7cad3bc4adf8594142c2d6a56206869c7029cb9c5493b8451001fc2f096ef9"}
Jul 14 11:08:20.288454 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:20.288101    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-htvd2"
Jul 14 11:08:20.289479 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:20.289454    2563 generic.go:356] "Generic (PLEG): container finished" podID="7e415343-0445-463e-9139-b8b47765a898" containerID="92c1cf20dcfa0388a43104e0c998c57e1cdd86318d91d44a698a96e418d9158c" exitCode=0
Jul 14 11:08:20.289573 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:20.289528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vpx7j" event={"ID":"7e415343-0445-463e-9139-b8b47765a898","Type":"ContainerDied","Data":"92c1cf20dcfa0388a43104e0c998c57e1cdd86318d91d44a698a96e418d9158c"}
Jul 14 11:08:20.314360 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:20.311590    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-htvd2" podStartSLOduration=67.000809646 podStartE2EDuration="1m9.31157056s" totalImagesPullingTime="2.310760914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:20.311090631 +0000 UTC m=+104.893629655" watchObservedRunningTime="2026-07-14 11:08:20.31157056 +0000 UTC m=+104.894109566"
Jul 14 11:08:21.295839 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.295802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:21.296262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.295975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vpx7j" event={"ID":"7e415343-0445-463e-9139-b8b47765a898","Type":"ContainerStarted","Data":"499135324a5fff07d0e0d5c3c4ea945f876ff89646ea415bf88e5c5faf266b0c"}
Jul 14 11:08:21.296262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.296064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vpx7j" event={"ID":"7e415343-0445-463e-9139-b8b47765a898","Type":"ContainerStarted","Data":"f8b41d95ec15afb701aae71c246d14976cc544af0a480980fc241254c39a2a87"}
Jul 14 11:08:21.315553 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.315507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vpx7j" podStartSLOduration=2.193018184 podStartE2EDuration="3.315493549s" totalImagesPullingTime="1.060823822s" totalInitContainerRuntime="61.651543ms" isStatefulPod=true podCreationTimestamp="2026-07-14 11:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:21.313796033 +0000 UTC m=+105.896335037" watchObservedRunningTime="2026-07-14 11:08:21.315493549 +0000 UTC m=+105.898032604"
Jul 14 11:08:21.395049 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.395017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:21.558724 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.558633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:21.996570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:21.996480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:23.057295 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:23.057258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:23.151554 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:23.151520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:23.299405 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:23.299379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:23.633887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:23.633856    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:23.735092 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:23.735061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:24.567177 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:24.567131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:24.670078 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:24.670047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:24.765024 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:24.764993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:26.904097 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:26.904067    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-657c88457b-mrjlh"]
Jul 14 11:08:26.993600 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:26.993567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:27.048759 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:27.048723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:27.223265 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:27.223178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:27.283386 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:27.283352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:28.987676 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:28.987628    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:29.050293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:29.050257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:30.298594 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:30.298559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-htvd2"
Jul 14 11:08:33.094273 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:33.094235    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:33.150566 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:33.150526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:33.581786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:33.581756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:33.682688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:33.682637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:34.144535 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:34.144505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:34.247346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:34.247310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:36.910173 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:36.910146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:37.289804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:37.289775    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:37.414995 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:37.414961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:39.490178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:39.490147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:42.164889 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:42.164850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:42.215257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:42.215222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:44.277403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:44.277374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:44.380796 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:44.380760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:44.479029 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:44.478995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:45.564165 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:45.564087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:08:45.566345 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:45.566324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558289b9-aa16-4008-a8fd-4eaa7271e009-metrics-certs\") pod \"network-metrics-daemon-xlw8z\" (UID: \"558289b9-aa16-4008-a8fd-4eaa7271e009\") " pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:08:45.841125 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:45.841044    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhbfh\""
Jul 14 11:08:45.849209 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:45.849189    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlw8z"
Jul 14 11:08:46.168835 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:46.168805    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlw8z"]
Jul 14 11:08:46.382232 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:46.382144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlw8z" event={"ID":"558289b9-aa16-4008-a8fd-4eaa7271e009","Type":"ContainerStarted","Data":"72725341e4a2e86ba581e9aba6d448702693d5475a0f5f6bee206906dfcea124"}
Jul 14 11:08:48.389835 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:48.389804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlw8z" event={"ID":"558289b9-aa16-4008-a8fd-4eaa7271e009","Type":"ContainerStarted","Data":"70d78a076fe4872e3f4af6267e40cf50e0aa1575d8c0d7a8c692700daf293984"}
Jul 14 11:08:48.390208 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:48.389853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlw8z" event={"ID":"558289b9-aa16-4008-a8fd-4eaa7271e009","Type":"ContainerStarted","Data":"fa37d75b4550ef6e36b9105b40dbef1612cb1fa2c6806871735b78a4e63f8530"}
Jul 14 11:08:48.409024 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:48.408979    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xlw8z" podStartSLOduration=130.928353337 podStartE2EDuration="2m12.408965953s" totalImagesPullingTime="1.480612616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:48.407744598 +0000 UTC m=+132.990283624" watchObservedRunningTime="2026-07-14 11:08:48.408965953 +0000 UTC m=+132.991504961"
Jul 14 11:08:48.516282 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:48.516231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:48.598797 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:48.598763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:51.611876 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:51.611844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:51.685810 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:51.685781    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:51.923352 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:51.923273    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-657c88457b-mrjlh" podUID="1c9fd828-d26d-4dca-b75a-16f200bec483" containerName="registry" containerID="cri-o://377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b" gracePeriod=30
Jul 14 11:08:52.161591 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.161566    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:52.312623 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c9fd828-d26d-4dca-b75a-16f200bec483-ca-trust-extracted\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.312623 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312628    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9twp\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-kube-api-access-b9twp\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.312887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312675    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-certificates\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.312887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-trusted-ca\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.312887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312740    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-installation-pull-secrets\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.312887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312848    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.313093 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-bound-sa-token\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.313093 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.312933    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-image-registry-private-configuration\") pod \"1c9fd828-d26d-4dca-b75a-16f200bec483\" (UID: \"1c9fd828-d26d-4dca-b75a-16f200bec483\") "
Jul 14 11:08:52.313235 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.313209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:08:52.313297 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.313234    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:08:52.315312 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.315276    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:08:52.315473 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.315439    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:08:52.315582 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.315492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:08:52.315582 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.315539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-kube-api-access-b9twp" (OuterVolumeSpecName: "kube-api-access-b9twp") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "kube-api-access-b9twp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:08:52.315767 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.315742    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:08:52.321856 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.321835    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9fd828-d26d-4dca-b75a-16f200bec483-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1c9fd828-d26d-4dca-b75a-16f200bec483" (UID: "1c9fd828-d26d-4dca-b75a-16f200bec483"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:08:52.404989 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.404961    2563 generic.go:356] "Generic (PLEG): container finished" podID="1c9fd828-d26d-4dca-b75a-16f200bec483" containerID="377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b" exitCode=0
Jul 14 11:08:52.405089 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.405011    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657c88457b-mrjlh"
Jul 14 11:08:52.405089 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.405041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657c88457b-mrjlh" event={"ID":"1c9fd828-d26d-4dca-b75a-16f200bec483","Type":"ContainerDied","Data":"377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b"}
Jul 14 11:08:52.405169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.405088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657c88457b-mrjlh" event={"ID":"1c9fd828-d26d-4dca-b75a-16f200bec483","Type":"ContainerDied","Data":"d5a16d56567bf51b8d3c9ef7be5883246c24b732bd527d32dfd8e594776009e2"}
Jul 14 11:08:52.405169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.405129    2563 scope.go:117] "RemoveContainer" containerID="377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b"
Jul 14 11:08:52.415956 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.415890    2563 scope.go:117] "RemoveContainer" containerID="377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b"
Jul 14 11:08:52.416106 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416084    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c9fd828-d26d-4dca-b75a-16f200bec483-ca-trust-extracted\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416112    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9twp\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-kube-api-access-b9twp\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416127    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-certificates\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416147    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c9fd828-d26d-4dca-b75a-16f200bec483-trusted-ca\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416161    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-installation-pull-secrets\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416174    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-registry-tls\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416188    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c9fd828-d26d-4dca-b75a-16f200bec483-bound-sa-token\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.416362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.416206    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c9fd828-d26d-4dca-b75a-16f200bec483-image-registry-private-configuration\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:08:52.420377 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:08:52.420328    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b\": container with ID starting with 377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b not found: ID does not exist" containerID="377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b"
Jul 14 11:08:52.420472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.420382    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b"} err="failed to get container status \"377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b\": rpc error: code = NotFound desc = could not find container \"377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b\": container with ID starting with 377d70e247c9d48f27e81993d54c4c163ab0406f4a19981933c3d312d540a92b not found: ID does not exist"
Jul 14 11:08:52.429195 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.429173    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-657c88457b-mrjlh"]
Jul 14 11:08:52.432930 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:52.432910    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-657c88457b-mrjlh"]
Jul 14 11:08:53.823596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:53.823565    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c9fd828-d26d-4dca-b75a-16f200bec483" path="/var/lib/kubelet/pods/1c9fd828-d26d-4dca-b75a-16f200bec483/volumes"
Jul 14 11:08:54.348377 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:54.348343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:08:57.422753 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:57.422721    2563 generic.go:356] "Generic (PLEG): container finished" podID="3da78d2b-82fb-4bcf-a7c7-518c8387914a" containerID="9b8dfd6123d418546dfa1011eeb434f633f54b4c630ea18e064085d9b2af27b8" exitCode=0
Jul 14 11:08:57.423120 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:57.422783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp" event={"ID":"3da78d2b-82fb-4bcf-a7c7-518c8387914a","Type":"ContainerDied","Data":"9b8dfd6123d418546dfa1011eeb434f633f54b4c630ea18e064085d9b2af27b8"}
Jul 14 11:08:57.423120 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:57.423074    2563 scope.go:117] "RemoveContainer" containerID="9b8dfd6123d418546dfa1011eeb434f633f54b4c630ea18e064085d9b2af27b8"
Jul 14 11:08:58.427595 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:08:58.427557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-qq2tp" event={"ID":"3da78d2b-82fb-4bcf-a7c7-518c8387914a","Type":"ContainerStarted","Data":"08d873612e9b767ecfe51af9fee95d7e5110e0a9ab893f5f935da2e5729238b7"}
Jul 14 11:09:01.232368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:01.232331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:01.303160 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:01.303122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:06.651059 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:06.651024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:06.706442 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:06.706406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:07.461790 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:07.461759    2563 generic.go:356] "Generic (PLEG): container finished" podID="5845ce2e-c307-4065-9a4e-78370d482b09" containerID="5897801a03f46185770d9b1e07acc7941f2bf1b43680fdb6a01622ba41860f59" exitCode=0
Jul 14 11:09:07.461946 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:07.461833    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk" event={"ID":"5845ce2e-c307-4065-9a4e-78370d482b09","Type":"ContainerDied","Data":"5897801a03f46185770d9b1e07acc7941f2bf1b43680fdb6a01622ba41860f59"}
Jul 14 11:09:07.462169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:07.462156    2563 scope.go:117] "RemoveContainer" containerID="5897801a03f46185770d9b1e07acc7941f2bf1b43680fdb6a01622ba41860f59"
Jul 14 11:09:08.467401 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:08.467370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cnggk" event={"ID":"5845ce2e-c307-4065-9a4e-78370d482b09","Type":"ContainerStarted","Data":"6764ed5ec89e742b2ffa65e4408e1e6f021d309b03e1ccffb608bc6313a21e0c"}
Jul 14 11:09:34.470583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:34.470487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:34.570633 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:34.570597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:46.382364 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:46.382312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:46.455332 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:46.455298    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:51.656071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:51.656038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:51.757833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:51.757803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:57.995673 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:57.995627    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:58.095987 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:58.095958    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:58.196514 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:58.196481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:58.735981 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:58.735941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:09:58.793833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:09:58.793801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:08.597637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:08.597599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:08.743471 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:08.743440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:23.043353 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:23.043319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:28.169836 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:28.169797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:28.223153 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:28.223119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:40.724010 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:40.723976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:40.887497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:40.887464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:44.566337 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:44.566292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:10:59.304977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:10:59.304892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:03.670236 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:03.670192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:04.227224 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:04.227186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:06.067795 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.067765    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"]
Jul 14 11:11:06.068155 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.068021    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c9fd828-d26d-4dca-b75a-16f200bec483" containerName="registry"
Jul 14 11:11:06.068155 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.068032    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c9fd828-d26d-4dca-b75a-16f200bec483" containerName="registry"
Jul 14 11:11:06.068155 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.068071    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c9fd828-d26d-4dca-b75a-16f200bec483" containerName="registry"
Jul 14 11:11:06.070700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.070684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.079004 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.078988    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 14 11:11:06.079079 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.079007    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:11:06.079129 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.079113    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pc4gw\""
Jul 14 11:11:06.103386 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.103365    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"]
Jul 14 11:11:06.135979 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.135950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctf57\" (UniqueName: \"kubernetes.io/projected/9706e496-a900-4db0-ad6e-84ae6862c76a-kube-api-access-ctf57\") pod \"cert-manager-operator-controller-manager-55c444bf86-z9g5d\" (UID: \"9706e496-a900-4db0-ad6e-84ae6862c76a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.136073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.135990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9706e496-a900-4db0-ad6e-84ae6862c76a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z9g5d\" (UID: \"9706e496-a900-4db0-ad6e-84ae6862c76a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.236828 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.236797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctf57\" (UniqueName: \"kubernetes.io/projected/9706e496-a900-4db0-ad6e-84ae6862c76a-kube-api-access-ctf57\") pod \"cert-manager-operator-controller-manager-55c444bf86-z9g5d\" (UID: \"9706e496-a900-4db0-ad6e-84ae6862c76a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.236979 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.236845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9706e496-a900-4db0-ad6e-84ae6862c76a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z9g5d\" (UID: \"9706e496-a900-4db0-ad6e-84ae6862c76a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.237216 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.237195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9706e496-a900-4db0-ad6e-84ae6862c76a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z9g5d\" (UID: \"9706e496-a900-4db0-ad6e-84ae6862c76a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.249884 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.249858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctf57\" (UniqueName: \"kubernetes.io/projected/9706e496-a900-4db0-ad6e-84ae6862c76a-kube-api-access-ctf57\") pod \"cert-manager-operator-controller-manager-55c444bf86-z9g5d\" (UID: \"9706e496-a900-4db0-ad6e-84ae6862c76a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.379315 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.379238    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"
Jul 14 11:11:06.550754 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:11:06.550714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9706e496_a900_4db0_ad6e_84ae6862c76a.slice/crio-9c47f54076a92d11a3121b6290d9ef1c01ca967f86acc3490d152354214230f0 WatchSource:0}: Error finding container 9c47f54076a92d11a3121b6290d9ef1c01ca967f86acc3490d152354214230f0: Status 404 returned error can't find the container with id 9c47f54076a92d11a3121b6290d9ef1c01ca967f86acc3490d152354214230f0
Jul 14 11:11:06.550875 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.550769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d"]
Jul 14 11:11:06.853755 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:06.853721    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d" event={"ID":"9706e496-a900-4db0-ad6e-84ae6862c76a","Type":"ContainerStarted","Data":"9c47f54076a92d11a3121b6290d9ef1c01ca967f86acc3490d152354214230f0"}
Jul 14 11:11:07.886272 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:07.886238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:08.082616 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:08.082575    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:09.866094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:09.866060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d" event={"ID":"9706e496-a900-4db0-ad6e-84ae6862c76a","Type":"ContainerStarted","Data":"aaa6d71f1f1ed953a608e7151855c3d91db4c7988d1c48490e52d55a88856460"}
Jul 14 11:11:09.903877 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:09.903826    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z9g5d" podStartSLOduration=1.3198138830000001 podStartE2EDuration="3.903810847s" totalImagesPullingTime="2.583996964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:09.901238712 +0000 UTC m=+274.483777718" watchObservedRunningTime="2026-07-14 11:11:09.903810847 +0000 UTC m=+274.486349851"
Jul 14 11:11:10.229842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:10.229771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:10.432097 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:10.432061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:10.631785 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:10.631737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:12.605498 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:12.605468    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:13.027435 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:13.027404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:14.004626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:14.004585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:14.084098 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:14.084071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:17.972300 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:17.972270    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h2nb2"]
Jul 14 11:11:17.975367 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:17.975350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:17.979068 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:17.979045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zr9qs\""
Jul 14 11:11:17.979200 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:17.979073    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 14 11:11:17.979810 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:17.979793    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 14 11:11:17.994022 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:17.994001    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h2nb2"]
Jul 14 11:11:18.017696 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.017673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51109726-c6c0-44c0-af47-99bb7fddc3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h2nb2\" (UID: \"51109726-c6c0-44c0-af47-99bb7fddc3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.017796 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.017712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4f8t\" (UniqueName: \"kubernetes.io/projected/51109726-c6c0-44c0-af47-99bb7fddc3f5-kube-api-access-q4f8t\") pod \"cert-manager-webhook-6486ffd997-h2nb2\" (UID: \"51109726-c6c0-44c0-af47-99bb7fddc3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.118988 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.118956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51109726-c6c0-44c0-af47-99bb7fddc3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h2nb2\" (UID: \"51109726-c6c0-44c0-af47-99bb7fddc3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.119116 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.119002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4f8t\" (UniqueName: \"kubernetes.io/projected/51109726-c6c0-44c0-af47-99bb7fddc3f5-kube-api-access-q4f8t\") pod \"cert-manager-webhook-6486ffd997-h2nb2\" (UID: \"51109726-c6c0-44c0-af47-99bb7fddc3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.131732 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.131700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51109726-c6c0-44c0-af47-99bb7fddc3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h2nb2\" (UID: \"51109726-c6c0-44c0-af47-99bb7fddc3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.136367 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.136345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4f8t\" (UniqueName: \"kubernetes.io/projected/51109726-c6c0-44c0-af47-99bb7fddc3f5-kube-api-access-q4f8t\") pod \"cert-manager-webhook-6486ffd997-h2nb2\" (UID: \"51109726-c6c0-44c0-af47-99bb7fddc3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.284243 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.284220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:18.405712 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.405687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h2nb2"]
Jul 14 11:11:18.408228 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:11:18.408201    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51109726_c6c0_44c0_af47_99bb7fddc3f5.slice/crio-ca4af4a3fed7998d2b33b19d7b2736ea2b49c9bda212d69d1dbfc63b9a4f175d WatchSource:0}: Error finding container ca4af4a3fed7998d2b33b19d7b2736ea2b49c9bda212d69d1dbfc63b9a4f175d: Status 404 returned error can't find the container with id ca4af4a3fed7998d2b33b19d7b2736ea2b49c9bda212d69d1dbfc63b9a4f175d
Jul 14 11:11:18.896696 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:18.896641    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2" event={"ID":"51109726-c6c0-44c0-af47-99bb7fddc3f5","Type":"ContainerStarted","Data":"ca4af4a3fed7998d2b33b19d7b2736ea2b49c9bda212d69d1dbfc63b9a4f175d"}
Jul 14 11:11:19.857902 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:19.857862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:21.803558 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.803522    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-v7kzj"]
Jul 14 11:11:21.806584 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.806568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:21.812762 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.812744    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sbngz\""
Jul 14 11:11:21.824205 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.824185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-v7kzj"]
Jul 14 11:11:21.849895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.849872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4xx8\" (UniqueName: \"kubernetes.io/projected/dda87f0b-62b8-42bd-a846-2dcd2227b617-kube-api-access-l4xx8\") pod \"cert-manager-55b4d954d5-v7kzj\" (UID: \"dda87f0b-62b8-42bd-a846-2dcd2227b617\") " pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:21.850012 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.849901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dda87f0b-62b8-42bd-a846-2dcd2227b617-bound-sa-token\") pod \"cert-manager-55b4d954d5-v7kzj\" (UID: \"dda87f0b-62b8-42bd-a846-2dcd2227b617\") " pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:21.909748 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.909720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2" event={"ID":"51109726-c6c0-44c0-af47-99bb7fddc3f5","Type":"ContainerStarted","Data":"5d30486c56fbc071bec144f4d84c1a32eb7cbb31307b6c74c36980fc77cd7558"}
Jul 14 11:11:21.909850 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.909802    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:21.944497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.944428    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2" podStartSLOduration=2.294395658 podStartE2EDuration="4.944411272s" totalImagesPullingTime="2.650015614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:21.943885519 +0000 UTC m=+286.526424521" watchObservedRunningTime="2026-07-14 11:11:21.944411272 +0000 UTC m=+286.526950289"
Jul 14 11:11:21.951199 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.951171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xx8\" (UniqueName: \"kubernetes.io/projected/dda87f0b-62b8-42bd-a846-2dcd2227b617-kube-api-access-l4xx8\") pod \"cert-manager-55b4d954d5-v7kzj\" (UID: \"dda87f0b-62b8-42bd-a846-2dcd2227b617\") " pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:21.951292 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.951205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dda87f0b-62b8-42bd-a846-2dcd2227b617-bound-sa-token\") pod \"cert-manager-55b4d954d5-v7kzj\" (UID: \"dda87f0b-62b8-42bd-a846-2dcd2227b617\") " pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:21.974584 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.974553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4xx8\" (UniqueName: \"kubernetes.io/projected/dda87f0b-62b8-42bd-a846-2dcd2227b617-kube-api-access-l4xx8\") pod \"cert-manager-55b4d954d5-v7kzj\" (UID: \"dda87f0b-62b8-42bd-a846-2dcd2227b617\") " pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:21.975339 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:21.975320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dda87f0b-62b8-42bd-a846-2dcd2227b617-bound-sa-token\") pod \"cert-manager-55b4d954d5-v7kzj\" (UID: \"dda87f0b-62b8-42bd-a846-2dcd2227b617\") " pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:22.114871 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:22.114800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-v7kzj"
Jul 14 11:11:22.239572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:22.239386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-v7kzj"]
Jul 14 11:11:22.914997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:22.914956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-v7kzj" event={"ID":"dda87f0b-62b8-42bd-a846-2dcd2227b617","Type":"ContainerStarted","Data":"e049d4bb7f51a924903c90620aebb4049546110589422f049b8cbd7d6d144a4b"}
Jul 14 11:11:22.914997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:22.914999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-v7kzj" event={"ID":"dda87f0b-62b8-42bd-a846-2dcd2227b617","Type":"ContainerStarted","Data":"b0e660b11734bab0c4e1496b0bf676bc65cfc0d741d7f494f7842feb639fa1f5"}
Jul 14 11:11:22.931344 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:22.931297    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-v7kzj" podStartSLOduration=1.9312828469999999 podStartE2EDuration="1.931282847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:22.930918395 +0000 UTC m=+287.513457422" watchObservedRunningTime="2026-07-14 11:11:22.931282847 +0000 UTC m=+287.513821851"
Jul 14 11:11:25.206050 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:25.206019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:25.300522 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:25.300478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:27.780171 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:27.780129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:27.917691 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:27.917646    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h2nb2"
Jul 14 11:11:27.935833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:27.935798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:31.541132 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:31.541093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:31.642778 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:31.642749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:33.225794 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:33.225764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:35.737270 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:35.737246    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:11:35.741360 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:35.741258    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 11:11:35.751105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:35.751080    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:11:43.975988 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:43.975958    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:44.079671 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:44.079635    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:45.781728 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:45.781694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:45.906701 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:45.906669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:46.429008 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:46.428978    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:46.740147 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:46.740065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:46.905535 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:46.905503    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:46.996564 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:46.996488    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"]
Jul 14 11:11:46.999250 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:46.999233    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.003861 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:11:47.003833    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret"
Jul 14 11:11:47.003861 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:11:47.003832    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 14 11:11:47.003861 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:11:47.003832    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 14 11:11:47.009597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.009579    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 11:11:47.009709 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.009611    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dzl9m\""
Jul 14 11:11:47.048205 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.048184    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"]
Jul 14 11:11:47.127448 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.127424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44b2011e-a297-4b0a-9c8f-4749b188b83f-manifests\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.127570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.127452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65sgb\" (UniqueName: \"kubernetes.io/projected/44b2011e-a297-4b0a-9c8f-4749b188b83f-kube-api-access-65sgb\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.127570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.127471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b2011e-a297-4b0a-9c8f-4749b188b83f-tmp\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.127570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.127488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44b2011e-a297-4b0a-9c8f-4749b188b83f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.127570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.127539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44b2011e-a297-4b0a-9c8f-4749b188b83f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.228638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.228610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44b2011e-a297-4b0a-9c8f-4749b188b83f-manifests\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.228638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.228643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65sgb\" (UniqueName: \"kubernetes.io/projected/44b2011e-a297-4b0a-9c8f-4749b188b83f-kube-api-access-65sgb\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.228843 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.228679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b2011e-a297-4b0a-9c8f-4749b188b83f-tmp\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.228843 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.228710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44b2011e-a297-4b0a-9c8f-4749b188b83f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.228843 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.228752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44b2011e-a297-4b0a-9c8f-4749b188b83f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.229075 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.229042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44b2011e-a297-4b0a-9c8f-4749b188b83f-manifests\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.230890 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.230869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b2011e-a297-4b0a-9c8f-4749b188b83f-tmp\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.231096 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.231079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44b2011e-a297-4b0a-9c8f-4749b188b83f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.231141 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.231109    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44b2011e-a297-4b0a-9c8f-4749b188b83f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:47.650942 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.650897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:47.777253 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.777223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:47.821519 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.821491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 11:11:47.861958 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.861929    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 11:11:47.867930 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.867903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:47.979997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:47.979909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:48.052664 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.052638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 11:11:48.064115 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.064093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65sgb\" (UniqueName: \"kubernetes.io/projected/44b2011e-a297-4b0a-9c8f-4749b188b83f-kube-api-access-65sgb\") pod \"opendatahub-operator-controller-manager-7ff488f585-rhh5r\" (UID: \"44b2011e-a297-4b0a-9c8f-4749b188b83f\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:48.098880 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.098850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:48.209428 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.209402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:48.240275 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.240205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:48.354735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.354705    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"]
Jul 14 11:11:48.358166 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:11:48.358141    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b2011e_a297_4b0a_9c8f_4749b188b83f.slice/crio-8ed8d7ff024f246ce387c66fec7a54ff7a7a37873206b3866f4237fec54a6d1e WatchSource:0}: Error finding container 8ed8d7ff024f246ce387c66fec7a54ff7a7a37873206b3866f4237fec54a6d1e: Status 404 returned error can't find the container with id 8ed8d7ff024f246ce387c66fec7a54ff7a7a37873206b3866f4237fec54a6d1e
Jul 14 11:11:48.359986 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.359971    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:11:48.362980 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:48.362924    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:49.011905 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:49.011868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r" event={"ID":"44b2011e-a297-4b0a-9c8f-4749b188b83f","Type":"ContainerStarted","Data":"8ed8d7ff024f246ce387c66fec7a54ff7a7a37873206b3866f4237fec54a6d1e"}
Jul 14 11:11:52.030404 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:52.030374    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r" event={"ID":"44b2011e-a297-4b0a-9c8f-4749b188b83f","Type":"ContainerStarted","Data":"2a061556406a60b24c2858cebce093a2523a4c4cbb0c0f530dc31c1d92615003"}
Jul 14 11:11:53.036554 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:53.036518    2563 generic.go:356] "Generic (PLEG): container finished" podID="44b2011e-a297-4b0a-9c8f-4749b188b83f" containerID="2a061556406a60b24c2858cebce093a2523a4c4cbb0c0f530dc31c1d92615003" exitCode=0
Jul 14 11:11:53.036972 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:53.036557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r" event={"ID":"44b2011e-a297-4b0a-9c8f-4749b188b83f","Type":"ContainerDied","Data":"2a061556406a60b24c2858cebce093a2523a4c4cbb0c0f530dc31c1d92615003"}
Jul 14 11:11:53.721411 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:53.721320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:53.873704 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:53.873646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:11:54.042100 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:54.042070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r" event={"ID":"44b2011e-a297-4b0a-9c8f-4749b188b83f","Type":"ContainerStarted","Data":"1fb8a49bc6bd858cf7c51e084956da857fb8c9b16a15ee2177603a13fcff52c8"}
Jul 14 11:11:54.042499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:54.042206    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:11:54.084862 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:11:54.084816    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r" podStartSLOduration=3.963868075 podStartE2EDuration="8.084803224s" totalImagesPullingTime="3.101138469s" totalInitContainerRuntime="1.01979668s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:54.084489551 +0000 UTC m=+318.667028568" watchObservedRunningTime="2026-07-14 11:11:54.084803224 +0000 UTC m=+318.667342229"
Jul 14 11:12:05.048298 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:05.048268    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-rhh5r"
Jul 14 11:12:07.178033 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:07.177992    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:08.392464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.392429    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9qchz"]
Jul 14 11:12:08.395563 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.395547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:08.421599 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.421573    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-n7x84\""
Jul 14 11:12:08.422066 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.422052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 14 11:12:08.429809 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.429787    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9qchz"]
Jul 14 11:12:08.484556 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.484527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tpzz\" (UniqueName: \"kubernetes.io/projected/a9eefb21-528a-4f63-b078-8f1032c8c5b7-kube-api-access-7tpzz\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:08.484711 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.484573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9eefb21-528a-4f63-b078-8f1032c8c5b7-cert\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:08.585194 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.585161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tpzz\" (UniqueName: \"kubernetes.io/projected/a9eefb21-528a-4f63-b078-8f1032c8c5b7-kube-api-access-7tpzz\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:08.585343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.585208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9eefb21-528a-4f63-b078-8f1032c8c5b7-cert\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:08.585343 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:08.585313    2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 14 11:12:08.585413 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:08.585364    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eefb21-528a-4f63-b078-8f1032c8c5b7-cert podName:a9eefb21-528a-4f63-b078-8f1032c8c5b7 nodeName:}" failed. No retries permitted until 2026-07-14 11:12:09.085348357 +0000 UTC m=+333.667887340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a9eefb21-528a-4f63-b078-8f1032c8c5b7-cert") pod "odh-model-controller-7998c67dc4-9qchz" (UID: "a9eefb21-528a-4f63-b078-8f1032c8c5b7") : secret "odh-model-controller-webhook-cert" not found
Jul 14 11:12:08.603316 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.603288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tpzz\" (UniqueName: \"kubernetes.io/projected/a9eefb21-528a-4f63-b078-8f1032c8c5b7-kube-api-access-7tpzz\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:08.909551 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:08.909515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:09.089788 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.089750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9eefb21-528a-4f63-b078-8f1032c8c5b7-cert\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:09.092120 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.092102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9eefb21-528a-4f63-b078-8f1032c8c5b7-cert\") pod \"odh-model-controller-7998c67dc4-9qchz\" (UID: \"a9eefb21-528a-4f63-b078-8f1032c8c5b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:09.305192 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.305166    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:09.463196 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.463151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9qchz"]
Jul 14 11:12:09.465773 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:09.465745    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9eefb21_528a_4f63_b078_8f1032c8c5b7.slice/crio-4a66db4805a376bf8d9da0ebfd6a5dcf46df4be60a2f66f2fccec88625c58af3 WatchSource:0}: Error finding container 4a66db4805a376bf8d9da0ebfd6a5dcf46df4be60a2f66f2fccec88625c58af3: Status 404 returned error can't find the container with id 4a66db4805a376bf8d9da0ebfd6a5dcf46df4be60a2f66f2fccec88625c58af3
Jul 14 11:12:09.563177 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.563086    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ck7lx"]
Jul 14 11:12:09.564024 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.563996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:09.567467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.567446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.570808 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.570790    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 14 11:12:09.694681 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.694636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0-tls-certs\") pod \"model-serving-api-54d89bd5fc-ck7lx\" (UID: \"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0\") " pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.694848 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.694698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqw88\" (UniqueName: \"kubernetes.io/projected/2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0-kube-api-access-xqw88\") pod \"model-serving-api-54d89bd5fc-ck7lx\" (UID: \"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0\") " pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.751080 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.751045    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ck7lx"]
Jul 14 11:12:09.795343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.795318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqw88\" (UniqueName: \"kubernetes.io/projected/2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0-kube-api-access-xqw88\") pod \"model-serving-api-54d89bd5fc-ck7lx\" (UID: \"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0\") " pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.795496 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.795393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0-tls-certs\") pod \"model-serving-api-54d89bd5fc-ck7lx\" (UID: \"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0\") " pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.797864 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.797837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0-tls-certs\") pod \"model-serving-api-54d89bd5fc-ck7lx\" (UID: \"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0\") " pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.816602 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.816543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqw88\" (UniqueName: \"kubernetes.io/projected/2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0-kube-api-access-xqw88\") pod \"model-serving-api-54d89bd5fc-ck7lx\" (UID: \"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0\") " pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:09.856415 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.856388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:09.881327 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:09.881302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:10.046773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:10.046749    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ck7lx"]
Jul 14 11:12:10.048442 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:10.048415    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f86a2c8_2bec_470e_9fcf_7fc78a36bdd0.slice/crio-31cd71463b61b96bea5a863ecbbf8bd930d0a3979603525ad4102f435fc2dbd3 WatchSource:0}: Error finding container 31cd71463b61b96bea5a863ecbbf8bd930d0a3979603525ad4102f435fc2dbd3: Status 404 returned error can't find the container with id 31cd71463b61b96bea5a863ecbbf8bd930d0a3979603525ad4102f435fc2dbd3
Jul 14 11:12:10.107428 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:10.107356    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz" event={"ID":"a9eefb21-528a-4f63-b078-8f1032c8c5b7","Type":"ContainerStarted","Data":"4a66db4805a376bf8d9da0ebfd6a5dcf46df4be60a2f66f2fccec88625c58af3"}
Jul 14 11:12:10.108386 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:10.108364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx" event={"ID":"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0","Type":"ContainerStarted","Data":"31cd71463b61b96bea5a863ecbbf8bd930d0a3979603525ad4102f435fc2dbd3"}
Jul 14 11:12:11.275356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.275306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:11.368896 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.368860    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f79597649-t4bls"]
Jul 14 11:12:11.376767 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.376741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.389138 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.389075    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 11:12:11.389138 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.389096    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 11:12:11.390206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.390189    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 11:12:11.390316 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.390189    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 11:12:11.401003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.400951    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7phsh\""
Jul 14 11:12:11.412293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.412236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f79597649-t4bls"]
Jul 14 11:12:11.510876 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.510840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/177c1b47-85e9-4844-9c70-a845b2d429d3-tls-certs\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.511073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.510891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g9g9\" (UniqueName: \"kubernetes.io/projected/177c1b47-85e9-4844-9c70-a845b2d429d3-kube-api-access-7g9g9\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.511073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.511000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/177c1b47-85e9-4844-9c70-a845b2d429d3-tmp\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.611771 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.611694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/177c1b47-85e9-4844-9c70-a845b2d429d3-tmp\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.611771 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.611743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/177c1b47-85e9-4844-9c70-a845b2d429d3-tls-certs\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.611771 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.611768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g9g9\" (UniqueName: \"kubernetes.io/projected/177c1b47-85e9-4844-9c70-a845b2d429d3-kube-api-access-7g9g9\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.612041 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:11.611865    2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 14 11:12:11.612041 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:11.611927    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/177c1b47-85e9-4844-9c70-a845b2d429d3-tls-certs podName:177c1b47-85e9-4844-9c70-a845b2d429d3 nodeName:}" failed. No retries permitted until 2026-07-14 11:12:12.111908398 +0000 UTC m=+336.694447397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/177c1b47-85e9-4844-9c70-a845b2d429d3-tls-certs") pod "kube-auth-proxy-7f79597649-t4bls" (UID: "177c1b47-85e9-4844-9c70-a845b2d429d3") : secret "kube-auth-proxy-tls" not found
Jul 14 11:12:11.614052 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.614025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/177c1b47-85e9-4844-9c70-a845b2d429d3-tmp\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.629311 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.629280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g9g9\" (UniqueName: \"kubernetes.io/projected/177c1b47-85e9-4844-9c70-a845b2d429d3-kube-api-access-7g9g9\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:11.770829 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:11.770788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:12.116877 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:12.116836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/177c1b47-85e9-4844-9c70-a845b2d429d3-tls-certs\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:12.119898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:12.119875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/177c1b47-85e9-4844-9c70-a845b2d429d3-tls-certs\") pod \"kube-auth-proxy-7f79597649-t4bls\" (UID: \"177c1b47-85e9-4844-9c70-a845b2d429d3\") " pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:12.287816 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:12.287781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls"
Jul 14 11:12:12.979302 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:12.979264    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f79597649-t4bls"]
Jul 14 11:12:12.981907 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:12.981883    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177c1b47_85e9_4844_9c70_a845b2d429d3.slice/crio-0c68d0bab24e62bb1606ecc304828bb0c5e2e74d9fbf36c0ce9d7ef6a6d22012 WatchSource:0}: Error finding container 0c68d0bab24e62bb1606ecc304828bb0c5e2e74d9fbf36c0ce9d7ef6a6d22012: Status 404 returned error can't find the container with id 0c68d0bab24e62bb1606ecc304828bb0c5e2e74d9fbf36c0ce9d7ef6a6d22012
Jul 14 11:12:13.125981 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.125892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz" event={"ID":"a9eefb21-528a-4f63-b078-8f1032c8c5b7","Type":"ContainerStarted","Data":"f04b8a0ce24db0b3ca16f0aeae72ad84b628fdb713ca3325e443f00bf4ecd0ef"}
Jul 14 11:12:13.126123 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.126005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:13.127165 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.127139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls" event={"ID":"177c1b47-85e9-4844-9c70-a845b2d429d3","Type":"ContainerStarted","Data":"0c68d0bab24e62bb1606ecc304828bb0c5e2e74d9fbf36c0ce9d7ef6a6d22012"}
Jul 14 11:12:13.131418 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.131389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx" event={"ID":"2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0","Type":"ContainerStarted","Data":"dc4078affb8234a8ef2d5284cd14617eadd9d957e46a252508873859646a9e94"}
Jul 14 11:12:13.131554 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.131536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:13.229232 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.229178    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz" podStartSLOduration=1.8444365230000002 podStartE2EDuration="5.229161884s" totalImagesPullingTime="3.384725361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:13.161357031 +0000 UTC m=+337.743896036" watchObservedRunningTime="2026-07-14 11:12:13.229161884 +0000 UTC m=+337.811700867"
Jul 14 11:12:13.229404 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.229292    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx" podStartSLOduration=1.427929451 podStartE2EDuration="4.229281456s" totalImagesPullingTime="2.801352005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:13.225057103 +0000 UTC m=+337.807596127" watchObservedRunningTime="2026-07-14 11:12:13.229281456 +0000 UTC m=+337.811820463"
Jul 14 11:12:13.395903 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.395824    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"]
Jul 14 11:12:13.399411 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.399389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.402203 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.402185    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 14 11:12:13.402556 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.402533    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7k8pq\""
Jul 14 11:12:13.402668 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.402621    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 14 11:12:13.402912 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.402900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 14 11:12:13.403095 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.403081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 14 11:12:13.405120 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.405079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:13.415328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.415304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"]
Jul 14 11:12:13.493282 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.493247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:13.529371 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a4daf-b2e6-41a8-8bce-8dda0e03428c-local-certs\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.529541 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.529541 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.529700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-token\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.529700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.529700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-cacerts\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.529700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.529686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6c6s\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-kube-api-access-x6c6s\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630295 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a4daf-b2e6-41a8-8bce-8dda0e03428c-local-certs\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-token\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-cacerts\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.630756 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.630497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6c6s\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-kube-api-access-x6c6s\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.631271 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.631241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.633131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.633106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a4daf-b2e6-41a8-8bce-8dda0e03428c-local-certs\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.633239 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.633143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.633294 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.633269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.633372 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.633350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-cacerts\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.638739 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.638720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-token\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.639056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.639034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6c6s\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-kube-api-access-x6c6s\") pod \"istiod-openshift-gateway-7574459656-t4khf\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.709996 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.709881    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:13.845494 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:13.845464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"]
Jul 14 11:12:13.847760 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:13.847730    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035a4daf_b2e6_41a8_8bce_8dda0e03428c.slice/crio-bd880f8442a41e06c89f016fab4505e8dbf0ad8011159c20df9a53f14d4c1313 WatchSource:0}: Error finding container bd880f8442a41e06c89f016fab4505e8dbf0ad8011159c20df9a53f14d4c1313: Status 404 returned error can't find the container with id bd880f8442a41e06c89f016fab4505e8dbf0ad8011159c20df9a53f14d4c1313
Jul 14 11:12:14.137194 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:14.137159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" event={"ID":"035a4daf-b2e6-41a8-8bce-8dda0e03428c","Type":"ContainerStarted","Data":"bd880f8442a41e06c89f016fab4505e8dbf0ad8011159c20df9a53f14d4c1313"}
Jul 14 11:12:14.812422 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:14.812387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:17.016282 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.016253    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:12:17.016565 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.016311    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:12:17.155148 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.155107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls" event={"ID":"177c1b47-85e9-4844-9c70-a845b2d429d3","Type":"ContainerStarted","Data":"cfb1b97840d589da4695ec9457d91df6996e3f44bbc85405e14525bf569bdec3"}
Jul 14 11:12:17.156677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.156626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" event={"ID":"035a4daf-b2e6-41a8-8bce-8dda0e03428c","Type":"ContainerStarted","Data":"910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b"}
Jul 14 11:12:17.156794 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.156693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:17.224558 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.224504    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f79597649-t4bls" podStartSLOduration=2.194657725 podStartE2EDuration="6.224489262s" totalImagesPullingTime="4.029831537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:17.222615287 +0000 UTC m=+341.805154349" watchObservedRunningTime="2026-07-14 11:12:17.224489262 +0000 UTC m=+341.807028266"
Jul 14 11:12:17.960311 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:17.960275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:18.162015 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:18.161980    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-t4khf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 14 11:12:18.162373 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:18.162042    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" podUID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 11:12:18.171789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:18.171757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:20.788779 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.788715    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" podStartSLOduration=4.622977617 podStartE2EDuration="7.788693532s" totalImagesPullingTime="3.165715915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:17.300794017 +0000 UTC m=+341.883333022" watchObservedRunningTime="2026-07-14 11:12:20.788693532 +0000 UTC m=+345.371232538"
Jul 14 11:12:20.790206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.790184    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"]
Jul 14 11:12:20.793952 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.793933    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.799840 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:20.799797    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="2f8ed243-8f8c-4bd8-9525-5e403805fa17" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.800393 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:20.800373    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"data-science-gateway-data-science-gateway-class-dockercfg-ms2gv\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ms2gv\"" type="*v1.Secret"
Jul 14 11:12:20.891263 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891355 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891355 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891355 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891469 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891469 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891537 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891537 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.891641 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.891615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbdrh\" (UniqueName: \"kubernetes.io/projected/2f8ed243-8f8c-4bd8-9525-5e403805fa17-kube-api-access-cbdrh\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.988767 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.988729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:20.992699 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992815 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992815 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992815 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992987 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbdrh\" (UniqueName: \"kubernetes.io/projected/2f8ed243-8f8c-4bd8-9525-5e403805fa17-kube-api-access-cbdrh\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992987 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992987 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992987 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.992987 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.992959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.993405 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.993380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.993555 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.993530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.993714 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.993690    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.993805 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.993723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.993991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.993969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.995502 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.995480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:20.995989 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:20.995969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:21.021840 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.021812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"]
Jul 14 11:12:21.052240 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.052190    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"]
Jul 14 11:12:21.057038 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.056971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.057847 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.057816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8ed243-8f8c-4bd8-9525-5e403805fa17-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:21.059163 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.059141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbdrh\" (UniqueName: \"kubernetes.io/projected/2f8ed243-8f8c-4bd8-9525-5e403805fa17-kube-api-access-cbdrh\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jm7x5\" (UID: \"2f8ed243-8f8c-4bd8-9525-5e403805fa17\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:21.086545 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.086529    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sv2z9\""
Jul 14 11:12:21.087112 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.087094    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 14 11:12:21.161751 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.161726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:21.180145 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.180120    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"]
Jul 14 11:12:21.183346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.183331    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.193971 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.193950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-kgxqw\" (UID: \"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.194063 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.194008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w4tq\" (UniqueName: \"kubernetes.io/projected/fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e-kube-api-access-5w4tq\") pod \"kserve-controller-manager-d6f4b65b8-kgxqw\" (UID: \"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.203637 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.203618    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 14 11:12:21.204716 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.204701    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dd6nf\""
Jul 14 11:12:21.229164 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.229139    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"]
Jul 14 11:12:21.242267 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.242242    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"]
Jul 14 11:12:21.294792 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.294761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f85bd830-b0f6-4acd-af45-9434c65e1b57-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.294949 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.294820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plqcq\" (UniqueName: \"kubernetes.io/projected/f85bd830-b0f6-4acd-af45-9434c65e1b57-kube-api-access-plqcq\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.294949 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.294856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w4tq\" (UniqueName: \"kubernetes.io/projected/fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e-kube-api-access-5w4tq\") pod \"kserve-controller-manager-d6f4b65b8-kgxqw\" (UID: \"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.295074 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.295015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-kgxqw\" (UID: \"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.297400 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.297380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e-cert\") pod \"kserve-controller-manager-d6f4b65b8-kgxqw\" (UID: \"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.377063 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.376991    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"]
Jul 14 11:12:21.377234 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.377211    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" podUID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" containerName="discovery" containerID="cri-o://910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b" gracePeriod=30
Jul 14 11:12:21.395762 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.395739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f85bd830-b0f6-4acd-af45-9434c65e1b57-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.396006 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.395772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plqcq\" (UniqueName: \"kubernetes.io/projected/f85bd830-b0f6-4acd-af45-9434c65e1b57-kube-api-access-plqcq\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.396527 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:21.396451    2563 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 14 11:12:21.396621 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:21.396611    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85bd830-b0f6-4acd-af45-9434c65e1b57-cert podName:f85bd830-b0f6-4acd-af45-9434c65e1b57 nodeName:}" failed. No retries permitted until 2026-07-14 11:12:21.896583938 +0000 UTC m=+346.479122938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f85bd830-b0f6-4acd-af45-9434c65e1b57-cert") pod "llmisvc-controller-manager-777c6bd7d5-lrbwz" (UID: "f85bd830-b0f6-4acd-af45-9434c65e1b57") : secret "llmisvc-webhook-server-cert" not found
Jul 14 11:12:21.453760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.453730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w4tq\" (UniqueName: \"kubernetes.io/projected/fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e-kube-api-access-5w4tq\") pod \"kserve-controller-manager-d6f4b65b8-kgxqw\" (UID: \"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.465088 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.465058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:21.501842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.501811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plqcq\" (UniqueName: \"kubernetes.io/projected/f85bd830-b0f6-4acd-af45-9434c65e1b57-kube-api-access-plqcq\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.608838 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.608814    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:21.663923 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.663854    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:21.678981 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.678957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:21.799583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799554    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a4daf-b2e6-41a8-8bce-8dda0e03428c-local-certs\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-dns-cert\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799618    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-cacerts\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-token\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6c6s\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-kube-api-access-x6c6s\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-ca-configmap\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.799871    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-kubeconfig\") pod \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\" (UID: \"035a4daf-b2e6-41a8-8bce-8dda0e03428c\") "
Jul 14 11:12:21.800496 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.800466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:12:21.801919 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.801892    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:12:21.802032 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.801943    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-token" (OuterVolumeSpecName: "istio-token") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:12:21.802098 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.802066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-cacerts" (OuterVolumeSpecName: "cacerts") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:12:21.802151 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.802112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035a4daf-b2e6-41a8-8bce-8dda0e03428c-local-certs" (OuterVolumeSpecName: "local-certs") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:12:21.802189 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.802158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-kube-api-access-x6c6s" (OuterVolumeSpecName: "kube-api-access-x6c6s") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "kube-api-access-x6c6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:12:21.802340 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.802322    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "035a4daf-b2e6-41a8-8bce-8dda0e03428c" (UID: "035a4daf-b2e6-41a8-8bce-8dda0e03428c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:12:21.812509 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.812481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:21.825716 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.825684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"]
Jul 14 11:12:21.828278 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:21.828258    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc032f67_d50d_4625_8ed7_e1c3a6d8ff2e.slice/crio-8cd106f7ec45869d6a795397642c08b29677dd8090e30a8b57f15a4af204fdb1 WatchSource:0}: Error finding container 8cd106f7ec45869d6a795397642c08b29677dd8090e30a8b57f15a4af204fdb1: Status 404 returned error can't find the container with id 8cd106f7ec45869d6a795397642c08b29677dd8090e30a8b57f15a4af204fdb1
Jul 14 11:12:21.901319 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f85bd830-b0f6-4acd-af45-9434c65e1b57-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901349    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-kubeconfig\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901360    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a4daf-b2e6-41a8-8bce-8dda0e03428c-local-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901369    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-dns-cert\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901377    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/035a4daf-b2e6-41a8-8bce-8dda0e03428c-cacerts\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901384    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-token\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901393    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6c6s\" (UniqueName: \"kubernetes.io/projected/035a4daf-b2e6-41a8-8bce-8dda0e03428c-kube-api-access-x6c6s\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.901444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.901401    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/035a4daf-b2e6-41a8-8bce-8dda0e03428c-istio-csr-ca-configmap\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:12:21.903532 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:21.903508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f85bd830-b0f6-4acd-af45-9434c65e1b57-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lrbwz\" (UID: \"f85bd830-b0f6-4acd-af45-9434c65e1b57\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:22.093044 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.092997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:22.107322 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.107264    2563 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 14 11:12:22.107486 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.107374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:22.180932 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.180861    2563 generic.go:356] "Generic (PLEG): container finished" podID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" containerID="910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b" exitCode=0
Jul 14 11:12:22.180932 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.180922    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"
Jul 14 11:12:22.181121 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.180937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" event={"ID":"035a4daf-b2e6-41a8-8bce-8dda0e03428c","Type":"ContainerDied","Data":"910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b"}
Jul 14 11:12:22.181121 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.180980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t4khf" event={"ID":"035a4daf-b2e6-41a8-8bce-8dda0e03428c","Type":"ContainerDied","Data":"bd880f8442a41e06c89f016fab4505e8dbf0ad8011159c20df9a53f14d4c1313"}
Jul 14 11:12:22.181121 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.180999    2563 scope.go:117] "RemoveContainer" containerID="910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b"
Jul 14 11:12:22.184329 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.184300    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw" event={"ID":"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e","Type":"ContainerStarted","Data":"8cd106f7ec45869d6a795397642c08b29677dd8090e30a8b57f15a4af204fdb1"}
Jul 14 11:12:22.193236 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.193215    2563 scope.go:117] "RemoveContainer" containerID="910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b"
Jul 14 11:12:22.193600 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:12:22.193527    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b\": container with ID starting with 910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b not found: ID does not exist" containerID="910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b"
Jul 14 11:12:22.193600 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.193565    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b"} err="failed to get container status \"910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b\": rpc error: code = NotFound desc = could not find container \"910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b\": container with ID starting with 910d82b910645fcc5c8aff5518878d3a09d476801c677ff25ab5e3a54a0d984b not found: ID does not exist"
Jul 14 11:12:22.217270 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.217245    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"]
Jul 14 11:12:22.233214 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.233188    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t4khf"]
Jul 14 11:12:22.256668 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.256632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"]
Jul 14 11:12:22.257875 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:22.257848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf85bd830_b0f6_4acd_af45_9434c65e1b57.slice/crio-cc39ab9743a9259a548c1855357b1ef3b1d8369083d418c1c0e691fa407baba7 WatchSource:0}: Error finding container cc39ab9743a9259a548c1855357b1ef3b1d8369083d418c1c0e691fa407baba7: Status 404 returned error can't find the container with id cc39ab9743a9259a548c1855357b1ef3b1d8369083d418c1c0e691fa407baba7
Jul 14 11:12:22.300830 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.300798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"]
Jul 14 11:12:22.301314 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:12:22.301292    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8ed243_8f8c_4bd8_9525_5e403805fa17.slice/crio-a4c52c0922fe9f0ec1d4e2a3c8e2a48fed5cf6584c9488f8822f6e55df5980cb WatchSource:0}: Error finding container a4c52c0922fe9f0ec1d4e2a3c8e2a48fed5cf6584c9488f8822f6e55df5980cb: Status 404 returned error can't find the container with id a4c52c0922fe9f0ec1d4e2a3c8e2a48fed5cf6584c9488f8822f6e55df5980cb
Jul 14 11:12:22.331403 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.331383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ms2gv\""
Jul 14 11:12:22.538295 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.538259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:22.694158 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:22.694122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:12:23.191983 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:23.191952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz" event={"ID":"f85bd830-b0f6-4acd-af45-9434c65e1b57","Type":"ContainerStarted","Data":"cc39ab9743a9259a548c1855357b1ef3b1d8369083d418c1c0e691fa407baba7"}
Jul 14 11:12:23.193284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:23.193251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5" event={"ID":"2f8ed243-8f8c-4bd8-9525-5e403805fa17","Type":"ContainerStarted","Data":"a4c52c0922fe9f0ec1d4e2a3c8e2a48fed5cf6584c9488f8822f6e55df5980cb"}
Jul 14 11:12:23.827073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:23.827035    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" path="/var/lib/kubelet/pods/035a4daf-b2e6-41a8-8bce-8dda0e03428c/volumes"
Jul 14 11:12:24.140053 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:24.139966    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-9qchz"
Jul 14 11:12:24.144141 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:24.144106    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ck7lx"
Jul 14 11:12:26.111238 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.111208    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:12:26.111564 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.111284    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:12:26.111564 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.111320    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:12:26.213085 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.213054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5" event={"ID":"2f8ed243-8f8c-4bd8-9525-5e403805fa17","Type":"ContainerStarted","Data":"74ad037c80017ae07458fd7e0cddb652c1879de213d927daeecc0eed2a94a3c5"}
Jul 14 11:12:26.214985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.214951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw" event={"ID":"fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e","Type":"ContainerStarted","Data":"dcd3cf36a8ce99587cea9713efcb41739900d211695db09004826af7350dfee8"}
Jul 14 11:12:26.215118 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.215014    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:12:26.216708 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.216680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz" event={"ID":"f85bd830-b0f6-4acd-af45-9434c65e1b57","Type":"ContainerStarted","Data":"7a5952bb62191485a4e32f16fedf8b811d6977ac66f91b14c67bb8fc6a241baa"}
Jul 14 11:12:26.216855 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.216826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:26.286760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.286695    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz" podStartSLOduration=1.4198873490000001 podStartE2EDuration="5.286678215s" totalImagesPullingTime="3.866790866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:26.286356388 +0000 UTC m=+350.868895416" watchObservedRunningTime="2026-07-14 11:12:26.286678215 +0000 UTC m=+350.869217217"
Jul 14 11:12:26.287129 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.287099    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5" podStartSLOduration=2.479213084 podStartE2EDuration="6.28708673s" totalImagesPullingTime="3.807873646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:26.247437157 +0000 UTC m=+350.829976251" watchObservedRunningTime="2026-07-14 11:12:26.28708673 +0000 UTC m=+350.869625734"
Jul 14 11:12:26.328208 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:26.328152    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw" podStartSLOduration=2.049746606 podStartE2EDuration="6.328141078s" totalImagesPullingTime="4.278394472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:26.32758325 +0000 UTC m=+350.910122308" watchObservedRunningTime="2026-07-14 11:12:26.328141078 +0000 UTC m=+350.910680082"
Jul 14 11:12:27.107576 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:27.107543    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:27.112352 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:27.112327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:27.220818 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:27.220791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:27.221884 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:27.221862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jm7x5"
Jul 14 11:12:57.223346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:57.223319    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lrbwz"
Jul 14 11:12:57.226691 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:12:57.226675    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kgxqw"
Jul 14 11:13:05.580192 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:05.580152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:08.680060 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.680027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:08.829100 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.829070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:08.907634 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.907601    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n9586"]
Jul 14 11:13:08.907935 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.907922    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" containerName="discovery"
Jul 14 11:13:08.907976 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.907936    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" containerName="discovery"
Jul 14 11:13:08.908008 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.907982    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="035a4daf-b2e6-41a8-8bce-8dda0e03428c" containerName="discovery"
Jul 14 11:13:08.910917 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.910903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:08.919944 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.919924    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 11:13:08.920630 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.920609    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nb9wv\""
Jul 14 11:13:08.921062 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.921034    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 11:13:08.927495 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.927474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n9586"]
Jul 14 11:13:08.965536 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:08.965453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdbjg\" (UniqueName: \"kubernetes.io/projected/66675814-1b04-4fcf-ac09-1089ba22b12f-kube-api-access-cdbjg\") pod \"kuadrant-operator-catalog-n9586\" (UID: \"66675814-1b04-4fcf-ac09-1089ba22b12f\") " pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:09.040075 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.040044    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:09.066122 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.066094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdbjg\" (UniqueName: \"kubernetes.io/projected/66675814-1b04-4fcf-ac09-1089ba22b12f-kube-api-access-cdbjg\") pod \"kuadrant-operator-catalog-n9586\" (UID: \"66675814-1b04-4fcf-ac09-1089ba22b12f\") " pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:09.076429 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.076401    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdbjg\" (UniqueName: \"kubernetes.io/projected/66675814-1b04-4fcf-ac09-1089ba22b12f-kube-api-access-cdbjg\") pod \"kuadrant-operator-catalog-n9586\" (UID: \"66675814-1b04-4fcf-ac09-1089ba22b12f\") " pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:09.193640 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.193608    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n9586"]
Jul 14 11:13:09.193879 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.193867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:09.324288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.324264    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n9586"]
Jul 14 11:13:09.326231 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:13:09.326208    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66675814_1b04_4fcf_ac09_1089ba22b12f.slice/crio-5eb303cf1e142459ebfafbddaa5a583eaeb39ff0d014ac872c538e74356c0b3a WatchSource:0}: Error finding container 5eb303cf1e142459ebfafbddaa5a583eaeb39ff0d014ac872c538e74356c0b3a: Status 404 returned error can't find the container with id 5eb303cf1e142459ebfafbddaa5a583eaeb39ff0d014ac872c538e74356c0b3a
Jul 14 11:13:09.393168 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.393136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n9586" event={"ID":"66675814-1b04-4fcf-ac09-1089ba22b12f","Type":"ContainerStarted","Data":"5eb303cf1e142459ebfafbddaa5a583eaeb39ff0d014ac872c538e74356c0b3a"}
Jul 14 11:13:09.405749 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.405727    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b4zdd"]
Jul 14 11:13:09.409931 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.409917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:09.421011 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.420990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b4zdd"]
Jul 14 11:13:09.470436 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.470406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmf2m\" (UniqueName: \"kubernetes.io/projected/2ba97b27-5bbe-4b3e-b34a-57be9eca86dc-kube-api-access-vmf2m\") pod \"kuadrant-operator-catalog-b4zdd\" (UID: \"2ba97b27-5bbe-4b3e-b34a-57be9eca86dc\") " pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:09.571004 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.570976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmf2m\" (UniqueName: \"kubernetes.io/projected/2ba97b27-5bbe-4b3e-b34a-57be9eca86dc-kube-api-access-vmf2m\") pod \"kuadrant-operator-catalog-b4zdd\" (UID: \"2ba97b27-5bbe-4b3e-b34a-57be9eca86dc\") " pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:09.579498 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.579471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmf2m\" (UniqueName: \"kubernetes.io/projected/2ba97b27-5bbe-4b3e-b34a-57be9eca86dc-kube-api-access-vmf2m\") pod \"kuadrant-operator-catalog-b4zdd\" (UID: \"2ba97b27-5bbe-4b3e-b34a-57be9eca86dc\") " pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:09.631303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.631278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:09.718732 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.718702    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:09.839710 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.839605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:09.859085 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:09.859058    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b4zdd"]
Jul 14 11:13:09.901263 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:13:09.901231    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba97b27_5bbe_4b3e_b34a_57be9eca86dc.slice/crio-f05e1485c224c240fd5b830ef7eccb114172c9ee78e8866196800a150fc11129 WatchSource:0}: Error finding container f05e1485c224c240fd5b830ef7eccb114172c9ee78e8866196800a150fc11129: Status 404 returned error can't find the container with id f05e1485c224c240fd5b830ef7eccb114172c9ee78e8866196800a150fc11129
Jul 14 11:13:10.399365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:10.399329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd" event={"ID":"2ba97b27-5bbe-4b3e-b34a-57be9eca86dc","Type":"ContainerStarted","Data":"f05e1485c224c240fd5b830ef7eccb114172c9ee78e8866196800a150fc11129"}
Jul 14 11:13:12.409723 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.409687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n9586" event={"ID":"66675814-1b04-4fcf-ac09-1089ba22b12f","Type":"ContainerStarted","Data":"4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9"}
Jul 14 11:13:12.410166 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.409752    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-n9586" podUID="66675814-1b04-4fcf-ac09-1089ba22b12f" containerName="registry-server" containerID="cri-o://4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9" gracePeriod=2
Jul 14 11:13:12.411232 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.411209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd" event={"ID":"2ba97b27-5bbe-4b3e-b34a-57be9eca86dc","Type":"ContainerStarted","Data":"26507d80eaa467d0ed0ffb4c1f9ebaca00ec21364c5e9226b054a4bec2a1ca9e"}
Jul 14 11:13:12.429441 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.429402    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n9586" podStartSLOduration=2.244225389 podStartE2EDuration="4.429389692s" totalImagesPullingTime="2.185164303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:13:12.428540931 +0000 UTC m=+397.011079935" watchObservedRunningTime="2026-07-14 11:13:12.429389692 +0000 UTC m=+397.011928697"
Jul 14 11:13:12.444942 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.444903    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd" podStartSLOduration=1.831552509 podStartE2EDuration="3.44489059s" totalImagesPullingTime="1.613338081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:13:12.444072023 +0000 UTC m=+397.026611028" watchObservedRunningTime="2026-07-14 11:13:12.44489059 +0000 UTC m=+397.027429595"
Jul 14 11:13:12.651822 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.651794    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:12.795621 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.795589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdbjg\" (UniqueName: \"kubernetes.io/projected/66675814-1b04-4fcf-ac09-1089ba22b12f-kube-api-access-cdbjg\") pod \"66675814-1b04-4fcf-ac09-1089ba22b12f\" (UID: \"66675814-1b04-4fcf-ac09-1089ba22b12f\") "
Jul 14 11:13:12.797791 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.797760    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66675814-1b04-4fcf-ac09-1089ba22b12f-kube-api-access-cdbjg" (OuterVolumeSpecName: "kube-api-access-cdbjg") pod "66675814-1b04-4fcf-ac09-1089ba22b12f" (UID: "66675814-1b04-4fcf-ac09-1089ba22b12f"). InnerVolumeSpecName "kube-api-access-cdbjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:13:12.897139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:12.897103    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdbjg\" (UniqueName: \"kubernetes.io/projected/66675814-1b04-4fcf-ac09-1089ba22b12f-kube-api-access-cdbjg\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:13:13.416362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.416325    2563 generic.go:356] "Generic (PLEG): container finished" podID="66675814-1b04-4fcf-ac09-1089ba22b12f" containerID="4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9" exitCode=0
Jul 14 11:13:13.416362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.416365    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n9586"
Jul 14 11:13:13.416867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.416403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n9586" event={"ID":"66675814-1b04-4fcf-ac09-1089ba22b12f","Type":"ContainerDied","Data":"4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9"}
Jul 14 11:13:13.416867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.416447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n9586" event={"ID":"66675814-1b04-4fcf-ac09-1089ba22b12f","Type":"ContainerDied","Data":"5eb303cf1e142459ebfafbddaa5a583eaeb39ff0d014ac872c538e74356c0b3a"}
Jul 14 11:13:13.416867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.416469    2563 scope.go:117] "RemoveContainer" containerID="4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9"
Jul 14 11:13:13.425675 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.425636    2563 scope.go:117] "RemoveContainer" containerID="4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9"
Jul 14 11:13:13.425920 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:13:13.425900    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9\": container with ID starting with 4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9 not found: ID does not exist" containerID="4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9"
Jul 14 11:13:13.425985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.425927    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9"} err="failed to get container status \"4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9\": rpc error: code = NotFound desc = could not find container \"4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9\": container with ID starting with 4320a3d025c38c74bb051d3ef21a21c03723f02b057bbf116e1b6698445e5bd9 not found: ID does not exist"
Jul 14 11:13:13.437303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.437279    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n9586"]
Jul 14 11:13:13.441504 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.441480    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n9586"]
Jul 14 11:13:13.824691 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:13.824645    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66675814-1b04-4fcf-ac09-1089ba22b12f" path="/var/lib/kubelet/pods/66675814-1b04-4fcf-ac09-1089ba22b12f/volumes"
Jul 14 11:13:19.718985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:19.718951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:19.719366 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:19.719015    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:19.741065 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:19.741039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:20.469932 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:20.469905    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-b4zdd"
Jul 14 11:13:33.174187 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:33.174132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:35.434788 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.434759    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"]
Jul 14 11:13:35.435158 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.435062    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66675814-1b04-4fcf-ac09-1089ba22b12f" containerName="registry-server"
Jul 14 11:13:35.435158 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.435074    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="66675814-1b04-4fcf-ac09-1089ba22b12f" containerName="registry-server"
Jul 14 11:13:35.435158 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.435146    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="66675814-1b04-4fcf-ac09-1089ba22b12f" containerName="registry-server"
Jul 14 11:13:35.443007 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.442985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:35.445718 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.445700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nppfm\""
Jul 14 11:13:35.452284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.452262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"]
Jul 14 11:13:35.474678 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.474380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2jr8\" (UniqueName: \"kubernetes.io/projected/8a93fb72-add5-46ea-b76a-49c490b1a6fc-kube-api-access-w2jr8\") pod \"limitador-operator-controller-manager-859d7b78bd-92dkl\" (UID: \"8a93fb72-add5-46ea-b76a-49c490b1a6fc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:35.574859 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.574824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2jr8\" (UniqueName: \"kubernetes.io/projected/8a93fb72-add5-46ea-b76a-49c490b1a6fc-kube-api-access-w2jr8\") pod \"limitador-operator-controller-manager-859d7b78bd-92dkl\" (UID: \"8a93fb72-add5-46ea-b76a-49c490b1a6fc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:35.583143 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.583115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2jr8\" (UniqueName: \"kubernetes.io/projected/8a93fb72-add5-46ea-b76a-49c490b1a6fc-kube-api-access-w2jr8\") pod \"limitador-operator-controller-manager-859d7b78bd-92dkl\" (UID: \"8a93fb72-add5-46ea-b76a-49c490b1a6fc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:35.755754 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.755692    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nppfm\""
Jul 14 11:13:35.763695 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.763678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:35.919231 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:35.919205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"]
Jul 14 11:13:35.922204 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:13:35.922175    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a93fb72_add5_46ea_b76a_49c490b1a6fc.slice/crio-d1708f462f7cb64b0e1a8ce9149d815c84c70234650311febb8468efe529ddeb WatchSource:0}: Error finding container d1708f462f7cb64b0e1a8ce9149d815c84c70234650311febb8468efe529ddeb: Status 404 returned error can't find the container with id d1708f462f7cb64b0e1a8ce9149d815c84c70234650311febb8468efe529ddeb
Jul 14 11:13:36.193162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:36.193130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:36.514733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:36.514639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl" event={"ID":"8a93fb72-add5-46ea-b76a-49c490b1a6fc","Type":"ContainerStarted","Data":"d1708f462f7cb64b0e1a8ce9149d815c84c70234650311febb8468efe529ddeb"}
Jul 14 11:13:37.000114 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:37.000079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:37.961210 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:37.961144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:38.060133 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:38.060088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:38.525221 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:38.525191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl" event={"ID":"8a93fb72-add5-46ea-b76a-49c490b1a6fc","Type":"ContainerStarted","Data":"6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3"}
Jul 14 11:13:38.525376 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:38.525322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:38.552820 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:38.552776    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl" podStartSLOduration=1.707624825 podStartE2EDuration="3.552761927s" totalImagesPullingTime="1.845137102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:13:38.551848918 +0000 UTC m=+423.134387924" watchObservedRunningTime="2026-07-14 11:13:38.552761927 +0000 UTC m=+423.135300932"
Jul 14 11:13:39.726930 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.726897    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"]
Jul 14 11:13:39.730280 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.730263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:39.733092 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.733072    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wsdsc\""
Jul 14 11:13:39.742578 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.742554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"]
Jul 14 11:13:39.806099 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.806070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9t4r\" (UniqueName: \"kubernetes.io/projected/3811920d-ef8b-4c59-8214-ab9889832643-kube-api-access-g9t4r\") pod \"authorino-operator-75cc8d5cb7-rm29r\" (UID: \"3811920d-ef8b-4c59-8214-ab9889832643\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:39.906416 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.906385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9t4r\" (UniqueName: \"kubernetes.io/projected/3811920d-ef8b-4c59-8214-ab9889832643-kube-api-access-g9t4r\") pod \"authorino-operator-75cc8d5cb7-rm29r\" (UID: \"3811920d-ef8b-4c59-8214-ab9889832643\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:39.920194 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:39.920161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9t4r\" (UniqueName: \"kubernetes.io/projected/3811920d-ef8b-4c59-8214-ab9889832643-kube-api-access-g9t4r\") pod \"authorino-operator-75cc8d5cb7-rm29r\" (UID: \"3811920d-ef8b-4c59-8214-ab9889832643\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:40.040106 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:40.040079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:40.167605 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:40.167579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"]
Jul 14 11:13:40.170209 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:13:40.170187    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3811920d_ef8b_4c59_8214_ab9889832643.slice/crio-ecc5647f564dc539501be826ea7a04b1d52ee72bd88748cb4f769e61d132e461 WatchSource:0}: Error finding container ecc5647f564dc539501be826ea7a04b1d52ee72bd88748cb4f769e61d132e461: Status 404 returned error can't find the container with id ecc5647f564dc539501be826ea7a04b1d52ee72bd88748cb4f769e61d132e461
Jul 14 11:13:40.425145 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:40.425056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:40.536235 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:40.536194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r" event={"ID":"3811920d-ef8b-4c59-8214-ab9889832643","Type":"ContainerStarted","Data":"ecc5647f564dc539501be826ea7a04b1d52ee72bd88748cb4f769e61d132e461"}
Jul 14 11:13:40.832479 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:40.832452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:42.230057 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:42.230018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:42.927787 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:42.927763    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"]
Jul 14 11:13:42.932289 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:42.932268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:42.935976 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:42.935954    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-phqq5\""
Jul 14 11:13:42.936131 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:42.936114    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 11:13:42.950875 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:42.950855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"]
Jul 14 11:13:43.028712 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.028681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8shc\" (UniqueName: \"kubernetes.io/projected/4a8ee53b-dcc1-41ba-9a61-e7078fa1e441-kube-api-access-q8shc\") pod \"dns-operator-controller-manager-864dd78c4c-j97kn\" (UID: \"4a8ee53b-dcc1-41ba-9a61-e7078fa1e441\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:43.129257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.129220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8shc\" (UniqueName: \"kubernetes.io/projected/4a8ee53b-dcc1-41ba-9a61-e7078fa1e441-kube-api-access-q8shc\") pod \"dns-operator-controller-manager-864dd78c4c-j97kn\" (UID: \"4a8ee53b-dcc1-41ba-9a61-e7078fa1e441\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:43.153488 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.153422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8shc\" (UniqueName: \"kubernetes.io/projected/4a8ee53b-dcc1-41ba-9a61-e7078fa1e441-kube-api-access-q8shc\") pod \"dns-operator-controller-manager-864dd78c4c-j97kn\" (UID: \"4a8ee53b-dcc1-41ba-9a61-e7078fa1e441\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:43.241530 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.241507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:43.373548 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.373524    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"]
Jul 14 11:13:43.375418 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:13:43.375392    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8ee53b_dcc1_41ba_9a61_e7078fa1e441.slice/crio-4db3e3a392dc1bf417223c59bf7d6ef65b1fd7d4af8df669695c2abd0c062daa WatchSource:0}: Error finding container 4db3e3a392dc1bf417223c59bf7d6ef65b1fd7d4af8df669695c2abd0c062daa: Status 404 returned error can't find the container with id 4db3e3a392dc1bf417223c59bf7d6ef65b1fd7d4af8df669695c2abd0c062daa
Jul 14 11:13:43.551311 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.551280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r" event={"ID":"3811920d-ef8b-4c59-8214-ab9889832643","Type":"ContainerStarted","Data":"80fe4295cced236fa26183347b3b56ab567a3a7e02f21ec22ed0673dcffcfaae"}
Jul 14 11:13:43.551465 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.551358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:43.552488 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.552468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn" event={"ID":"4a8ee53b-dcc1-41ba-9a61-e7078fa1e441","Type":"ContainerStarted","Data":"4db3e3a392dc1bf417223c59bf7d6ef65b1fd7d4af8df669695c2abd0c062daa"}
Jul 14 11:13:43.574081 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.574042    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r" podStartSLOduration=1.851768858 podStartE2EDuration="4.574024858s" totalImagesPullingTime="2.722256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:13:43.57324658 +0000 UTC m=+428.155785611" watchObservedRunningTime="2026-07-14 11:13:43.574024858 +0000 UTC m=+428.156563863"
Jul 14 11:13:43.788035 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:43.788007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:44.548346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:44.548308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:46.567580 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:46.567543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn" event={"ID":"4a8ee53b-dcc1-41ba-9a61-e7078fa1e441","Type":"ContainerStarted","Data":"e476d8777dd2a8daf1818200fcd8bfeb5d942bf8cda05b4e43276677d00f4823"}
Jul 14 11:13:46.567974 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:46.567609    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:46.608104 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:46.608058    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn" podStartSLOduration=2.2854609359999998 podStartE2EDuration="4.60804285s" totalImagesPullingTime="2.322581914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:13:46.606207095 +0000 UTC m=+431.188746099" watchObservedRunningTime="2026-07-14 11:13:46.60804285 +0000 UTC m=+431.190581855"
Jul 14 11:13:47.828473 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:47.828440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:48.356475 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.356445    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"]
Jul 14 11:13:48.359775 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.359754    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.362631 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.362607    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dzxhj\""
Jul 14 11:13:48.373062 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.373042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d79f852b-844b-4f48-aba9-9abf63dc88b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.373144 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.373069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9jl2\" (UniqueName: \"kubernetes.io/projected/d79f852b-844b-4f48-aba9-9abf63dc88b9-kube-api-access-h9jl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.388553 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.388530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"]
Jul 14 11:13:48.473773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.473746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d79f852b-844b-4f48-aba9-9abf63dc88b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.473932 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.473780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9jl2\" (UniqueName: \"kubernetes.io/projected/d79f852b-844b-4f48-aba9-9abf63dc88b9-kube-api-access-h9jl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.474112 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.474094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d79f852b-844b-4f48-aba9-9abf63dc88b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.484972 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.484945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9jl2\" (UniqueName: \"kubernetes.io/projected/d79f852b-844b-4f48-aba9-9abf63dc88b9-kube-api-access-h9jl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.669718 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.669633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:48.824012 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:48.823987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"]
Jul 14 11:13:48.827378 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:13:48.827349    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79f852b_844b_4f48_aba9_9abf63dc88b9.slice/crio-08755df277d8b461b30d80e16fc0b131fb11028d6dbcae65c0fdc18f9e9e0019 WatchSource:0}: Error finding container 08755df277d8b461b30d80e16fc0b131fb11028d6dbcae65c0fdc18f9e9e0019: Status 404 returned error can't find the container with id 08755df277d8b461b30d80e16fc0b131fb11028d6dbcae65c0fdc18f9e9e0019
Jul 14 11:13:49.340117 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:49.340078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:49.459508 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:49.459460    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:49.532067 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:49.532041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:13:49.581483 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:49.581455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2" event={"ID":"d79f852b-844b-4f48-aba9-9abf63dc88b9","Type":"ContainerStarted","Data":"08755df277d8b461b30d80e16fc0b131fb11028d6dbcae65c0fdc18f9e9e0019"}
Jul 14 11:13:50.010052 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:50.010017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:50.117574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:50.117529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:53.991161 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:53.991123    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:54.091009 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:54.090979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:54.195105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:54.195065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:54.297354 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:54.297308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:54.560215 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:54.560142    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rm29r"
Jul 14 11:13:56.616833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:56.616804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2" event={"ID":"d79f852b-844b-4f48-aba9-9abf63dc88b9","Type":"ContainerStarted","Data":"0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3"}
Jul 14 11:13:56.617200 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:56.616879    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:13:56.657492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:56.657445    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2" podStartSLOduration=1.232512704 podStartE2EDuration="8.657429992s" totalImagesPullingTime="7.424917288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:13:56.654511588 +0000 UTC m=+441.237050628" watchObservedRunningTime="2026-07-14 11:13:56.657429992 +0000 UTC m=+441.239968996"
Jul 14 11:13:57.574436 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:57.574406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j97kn"
Jul 14 11:13:57.849145 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:57.849024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:58.019238 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:58.019205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:13:59.545964 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:13:59.545928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:00.171620 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:00.171589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:02.435675 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:02.435626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:02.541383 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:02.541354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:07.623549 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:07.623521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:09.496536 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.496475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:09.612938 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.612906    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"]
Jul 14 11:14:09.613135 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.613110    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" containerName="manager" containerID="cri-o://0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3" gracePeriod=2
Jul 14 11:14:09.615497 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:09.615436    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:09.615634 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.615558    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"]
Jul 14 11:14:09.641832 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.641800    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"]
Jul 14 11:14:09.642088 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.642037    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" containerName="manager" containerID="cri-o://6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3" gracePeriod=2
Jul 14 11:14:09.651587 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.651560    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"]
Jul 14 11:14:09.651994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.651979    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" containerName="manager"
Jul 14 11:14:09.651994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.651996    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" containerName="manager"
Jul 14 11:14:09.652095 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.652061    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" containerName="manager"
Jul 14 11:14:09.655526 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.655508    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.663761 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:09.663720    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:09.664934 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.664915    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"]
Jul 14 11:14:09.665950 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:09.665918    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-92dkl\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:14:09.668777 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:09.668752    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:09.685351 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.685329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:09.685758 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.685738    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"]
Jul 14 11:14:09.699301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.699282    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"]
Jul 14 11:14:09.699613 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.699598    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" containerName="manager"
Jul 14 11:14:09.699677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.699617    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" containerName="manager"
Jul 14 11:14:09.699747 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.699734    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" containerName="manager"
Jul 14 11:14:09.703154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.703139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:09.732079 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.732060    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"]
Jul 14 11:14:09.737484 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.737462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa8f654-ac13-4f83-bada-2faf03155f1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b7d5q\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.737580 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.737490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4fnc\" (UniqueName: \"kubernetes.io/projected/dfa8f654-ac13-4f83-bada-2faf03155f1c-kube-api-access-t4fnc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b7d5q\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.764207 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:09.764175    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-92dkl\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:14:09.767577 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:09.767543    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tzgr2\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:09.838395 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.838301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa8f654-ac13-4f83-bada-2faf03155f1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b7d5q\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.838395 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.838334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4fnc\" (UniqueName: \"kubernetes.io/projected/dfa8f654-ac13-4f83-bada-2faf03155f1c-kube-api-access-t4fnc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b7d5q\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.838395 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.838387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkhlg\" (UniqueName: \"kubernetes.io/projected/f8569c6f-8fd6-4487-b579-bf30da3823ef-kube-api-access-kkhlg\") pod \"limitador-operator-controller-manager-859d7b78bd-9vv54\" (UID: \"f8569c6f-8fd6-4487-b579-bf30da3823ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:09.838818 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.838675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa8f654-ac13-4f83-bada-2faf03155f1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b7d5q\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.856643 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.856616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4fnc\" (UniqueName: \"kubernetes.io/projected/dfa8f654-ac13-4f83-bada-2faf03155f1c-kube-api-access-t4fnc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b7d5q\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:09.894081 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.894057    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:09.896730 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.896713    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:14:09.939543 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.939513    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2jr8\" (UniqueName: \"kubernetes.io/projected/8a93fb72-add5-46ea-b76a-49c490b1a6fc-kube-api-access-w2jr8\") pod \"8a93fb72-add5-46ea-b76a-49c490b1a6fc\" (UID: \"8a93fb72-add5-46ea-b76a-49c490b1a6fc\") "
Jul 14 11:14:09.939643 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.939558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d79f852b-844b-4f48-aba9-9abf63dc88b9-extensions-socket-volume\") pod \"d79f852b-844b-4f48-aba9-9abf63dc88b9\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") "
Jul 14 11:14:09.939643 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.939582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9jl2\" (UniqueName: \"kubernetes.io/projected/d79f852b-844b-4f48-aba9-9abf63dc88b9-kube-api-access-h9jl2\") pod \"d79f852b-844b-4f48-aba9-9abf63dc88b9\" (UID: \"d79f852b-844b-4f48-aba9-9abf63dc88b9\") "
Jul 14 11:14:09.939898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.939872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkhlg\" (UniqueName: \"kubernetes.io/projected/f8569c6f-8fd6-4487-b579-bf30da3823ef-kube-api-access-kkhlg\") pod \"limitador-operator-controller-manager-859d7b78bd-9vv54\" (UID: \"f8569c6f-8fd6-4487-b579-bf30da3823ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:09.940104 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.940077    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d79f852b-844b-4f48-aba9-9abf63dc88b9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d79f852b-844b-4f48-aba9-9abf63dc88b9" (UID: "d79f852b-844b-4f48-aba9-9abf63dc88b9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:14:09.941583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.941563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79f852b-844b-4f48-aba9-9abf63dc88b9-kube-api-access-h9jl2" (OuterVolumeSpecName: "kube-api-access-h9jl2") pod "d79f852b-844b-4f48-aba9-9abf63dc88b9" (UID: "d79f852b-844b-4f48-aba9-9abf63dc88b9"). InnerVolumeSpecName "kube-api-access-h9jl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:14:09.941649 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.941587    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a93fb72-add5-46ea-b76a-49c490b1a6fc-kube-api-access-w2jr8" (OuterVolumeSpecName: "kube-api-access-w2jr8") pod "8a93fb72-add5-46ea-b76a-49c490b1a6fc" (UID: "8a93fb72-add5-46ea-b76a-49c490b1a6fc"). InnerVolumeSpecName "kube-api-access-w2jr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:14:09.950877 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:09.950858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkhlg\" (UniqueName: \"kubernetes.io/projected/f8569c6f-8fd6-4487-b579-bf30da3823ef-kube-api-access-kkhlg\") pod \"limitador-operator-controller-manager-859d7b78bd-9vv54\" (UID: \"f8569c6f-8fd6-4487-b579-bf30da3823ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:10.032702 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.032677    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:10.041022 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.040997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:10.041233 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.041213    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2jr8\" (UniqueName: \"kubernetes.io/projected/8a93fb72-add5-46ea-b76a-49c490b1a6fc-kube-api-access-w2jr8\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:14:10.041293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.041240    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d79f852b-844b-4f48-aba9-9abf63dc88b9-extensions-socket-volume\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:14:10.041293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.041254    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9jl2\" (UniqueName: \"kubernetes.io/projected/d79f852b-844b-4f48-aba9-9abf63dc88b9-kube-api-access-h9jl2\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:14:10.181337 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.181244    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"]
Jul 14 11:14:10.184238 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:14:10.184211    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa8f654_ac13_4f83_bada_2faf03155f1c.slice/crio-d5744a9c0c71329b10d86830d0afb62f0175990501e6783a37999574b0fffd00 WatchSource:0}: Error finding container d5744a9c0c71329b10d86830d0afb62f0175990501e6783a37999574b0fffd00: Status 404 returned error can't find the container with id d5744a9c0c71329b10d86830d0afb62f0175990501e6783a37999574b0fffd00
Jul 14 11:14:10.200977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.200955    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"]
Jul 14 11:14:10.202692 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:14:10.202667    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8569c6f_8fd6_4487_b579_bf30da3823ef.slice/crio-f3619f87a444f078bdf783f0bd7eaf6d642416a973da06ae7eb2b79aec44a165 WatchSource:0}: Error finding container f3619f87a444f078bdf783f0bd7eaf6d642416a973da06ae7eb2b79aec44a165: Status 404 returned error can't find the container with id f3619f87a444f078bdf783f0bd7eaf6d642416a973da06ae7eb2b79aec44a165
Jul 14 11:14:10.681328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.681296    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" containerID="6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3" exitCode=0
Jul 14 11:14:10.681747 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.681338    2563 scope.go:117] "RemoveContainer" containerID="6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3"
Jul 14 11:14:10.681747 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.681342    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-92dkl"
Jul 14 11:14:10.683248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.683224    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q" event={"ID":"dfa8f654-ac13-4f83-bada-2faf03155f1c","Type":"ContainerStarted","Data":"85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747"}
Jul 14 11:14:10.683390 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.683271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q" event={"ID":"dfa8f654-ac13-4f83-bada-2faf03155f1c","Type":"ContainerStarted","Data":"d5744a9c0c71329b10d86830d0afb62f0175990501e6783a37999574b0fffd00"}
Jul 14 11:14:10.683465 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.683443    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:10.684849 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.684817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54" event={"ID":"f8569c6f-8fd6-4487-b579-bf30da3823ef","Type":"ContainerStarted","Data":"a24dcd7c296b7782badf6101fdf52c11e75cc02e584bb01ee0fb4e9fd5b2e32b"}
Jul 14 11:14:10.684953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.684871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54" event={"ID":"f8569c6f-8fd6-4487-b579-bf30da3823ef","Type":"ContainerStarted","Data":"f3619f87a444f078bdf783f0bd7eaf6d642416a973da06ae7eb2b79aec44a165"}
Jul 14 11:14:10.684953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.684902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:10.686103 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.686076    2563 generic.go:356] "Generic (PLEG): container finished" podID="d79f852b-844b-4f48-aba9-9abf63dc88b9" containerID="0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3" exitCode=0
Jul 14 11:14:10.686195 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.686105    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzgr2"
Jul 14 11:14:10.690932 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.690915    2563 scope.go:117] "RemoveContainer" containerID="6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3"
Jul 14 11:14:10.691186 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:10.691166    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3\": container with ID starting with 6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3 not found: ID does not exist" containerID="6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3"
Jul 14 11:14:10.691250 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.691196    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3"} err="failed to get container status \"6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3\": rpc error: code = NotFound desc = could not find container \"6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3\": container with ID starting with 6542d25fbb9d01c56024c7de2627bd1a0f87035b04a70f5a9aca29ae5b95e5a3 not found: ID does not exist"
Jul 14 11:14:10.691250 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.691219    2563 scope.go:117] "RemoveContainer" containerID="0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3"
Jul 14 11:14:10.698745 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.698728    2563 scope.go:117] "RemoveContainer" containerID="0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3"
Jul 14 11:14:10.699001 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:10.698986    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3\": container with ID starting with 0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3 not found: ID does not exist" containerID="0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3"
Jul 14 11:14:10.699050 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.699007    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3"} err="failed to get container status \"0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3\": rpc error: code = NotFound desc = could not find container \"0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3\": container with ID starting with 0a2b0e01a51f7d49a700726f6fc9bd65ce35858405fec45bdbd0817a092317d3 not found: ID does not exist"
Jul 14 11:14:10.710871 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.710833    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q" podStartSLOduration=1.710818367 podStartE2EDuration="1.710818367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:14:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:14:10.708552886 +0000 UTC m=+455.291091913" watchObservedRunningTime="2026-07-14 11:14:10.710818367 +0000 UTC m=+455.293357373"
Jul 14 11:14:10.730639 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.730601    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54" podStartSLOduration=1.7305935099999998 podStartE2EDuration="1.73059351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:14:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:14:10.729698615 +0000 UTC m=+455.312237635" watchObservedRunningTime="2026-07-14 11:14:10.73059351 +0000 UTC m=+455.313132515"
Jul 14 11:14:10.815795 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.815764    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"]
Jul 14 11:14:10.820813 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.820798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:10.835313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.835294    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"]
Jul 14 11:14:10.946668 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.946577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8968d\" (UniqueName: \"kubernetes.io/projected/6e8f61dd-d325-4f38-8bc7-55926cc48a55-kube-api-access-8968d\") pod \"kuadrant-operator-controller-manager-86878c769f-8btxs\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:10.946797 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:10.946647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e8f61dd-d325-4f38-8bc7-55926cc48a55-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8btxs\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:11.047701 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.047648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8968d\" (UniqueName: \"kubernetes.io/projected/6e8f61dd-d325-4f38-8bc7-55926cc48a55-kube-api-access-8968d\") pod \"kuadrant-operator-controller-manager-86878c769f-8btxs\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:11.047846 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.047733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e8f61dd-d325-4f38-8bc7-55926cc48a55-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8btxs\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:11.048103 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.048086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e8f61dd-d325-4f38-8bc7-55926cc48a55-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8btxs\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:11.057253 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.057228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8968d\" (UniqueName: \"kubernetes.io/projected/6e8f61dd-d325-4f38-8bc7-55926cc48a55-kube-api-access-8968d\") pod \"kuadrant-operator-controller-manager-86878c769f-8btxs\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:11.130140 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.130091    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:11.310368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.310265    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"]
Jul 14 11:14:11.312890 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:14:11.312859    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8f61dd_d325_4f38_8bc7_55926cc48a55.slice/crio-22a0168443b3f46aa66d41c19265d8f9bd197bac68fe2b04c9b08853cfe84cc1 WatchSource:0}: Error finding container 22a0168443b3f46aa66d41c19265d8f9bd197bac68fe2b04c9b08853cfe84cc1: Status 404 returned error can't find the container with id 22a0168443b3f46aa66d41c19265d8f9bd197bac68fe2b04c9b08853cfe84cc1
Jul 14 11:14:11.693639 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.693604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs" event={"ID":"6e8f61dd-d325-4f38-8bc7-55926cc48a55","Type":"ContainerStarted","Data":"11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb"}
Jul 14 11:14:11.694033 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.693682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs" event={"ID":"6e8f61dd-d325-4f38-8bc7-55926cc48a55","Type":"ContainerStarted","Data":"22a0168443b3f46aa66d41c19265d8f9bd197bac68fe2b04c9b08853cfe84cc1"}
Jul 14 11:14:11.723523 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.723470    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs" podStartSLOduration=1.723450894 podStartE2EDuration="1.723450894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:14:11.721387571 +0000 UTC m=+456.303926588" watchObservedRunningTime="2026-07-14 11:14:11.723450894 +0000 UTC m=+456.305989900"
Jul 14 11:14:11.825727 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.825696    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a93fb72-add5-46ea-b76a-49c490b1a6fc" path="/var/lib/kubelet/pods/8a93fb72-add5-46ea-b76a-49c490b1a6fc/volumes"
Jul 14 11:14:11.826032 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:11.826019    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d79f852b-844b-4f48-aba9-9abf63dc88b9" path="/var/lib/kubelet/pods/d79f852b-844b-4f48-aba9-9abf63dc88b9/volumes"
Jul 14 11:14:12.698287 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:12.698257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:12.953904 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:12.953816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:13.051536 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:13.051503    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:13.172290 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:13.172252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:13.294783 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:13.294752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:16.695318 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:16.695277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:16.774432 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:16.774399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:21.416727 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:21.416677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:21.517460 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:21.517426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:21.696056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:21.695978    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9vv54"
Jul 14 11:14:21.696056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:21.696056    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:23.706555 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:23.706521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:14:23.764596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:23.764564    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"]
Jul 14 11:14:23.764776 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:23.764753    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q" podUID="dfa8f654-ac13-4f83-bada-2faf03155f1c" containerName="manager" containerID="cri-o://85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747" gracePeriod=10
Jul 14 11:14:24.016887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.016866    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:24.152382 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.152354    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa8f654-ac13-4f83-bada-2faf03155f1c-extensions-socket-volume\") pod \"dfa8f654-ac13-4f83-bada-2faf03155f1c\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") "
Jul 14 11:14:24.152536 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.152412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4fnc\" (UniqueName: \"kubernetes.io/projected/dfa8f654-ac13-4f83-bada-2faf03155f1c-kube-api-access-t4fnc\") pod \"dfa8f654-ac13-4f83-bada-2faf03155f1c\" (UID: \"dfa8f654-ac13-4f83-bada-2faf03155f1c\") "
Jul 14 11:14:24.152727 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.152704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa8f654-ac13-4f83-bada-2faf03155f1c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dfa8f654-ac13-4f83-bada-2faf03155f1c" (UID: "dfa8f654-ac13-4f83-bada-2faf03155f1c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:14:24.154372 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.154346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa8f654-ac13-4f83-bada-2faf03155f1c-kube-api-access-t4fnc" (OuterVolumeSpecName: "kube-api-access-t4fnc") pod "dfa8f654-ac13-4f83-bada-2faf03155f1c" (UID: "dfa8f654-ac13-4f83-bada-2faf03155f1c"). InnerVolumeSpecName "kube-api-access-t4fnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:14:24.253839 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.253782    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfa8f654-ac13-4f83-bada-2faf03155f1c-extensions-socket-volume\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:14:24.253839 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.253802    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4fnc\" (UniqueName: \"kubernetes.io/projected/dfa8f654-ac13-4f83-bada-2faf03155f1c-kube-api-access-t4fnc\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:14:24.760197 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.760157    2563 generic.go:356] "Generic (PLEG): container finished" podID="dfa8f654-ac13-4f83-bada-2faf03155f1c" containerID="85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747" exitCode=0
Jul 14 11:14:24.760589 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.760220    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"
Jul 14 11:14:24.760589 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.760237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q" event={"ID":"dfa8f654-ac13-4f83-bada-2faf03155f1c","Type":"ContainerDied","Data":"85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747"}
Jul 14 11:14:24.760589 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.760314    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q" event={"ID":"dfa8f654-ac13-4f83-bada-2faf03155f1c","Type":"ContainerDied","Data":"d5744a9c0c71329b10d86830d0afb62f0175990501e6783a37999574b0fffd00"}
Jul 14 11:14:24.760589 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.760335    2563 scope.go:117] "RemoveContainer" containerID="85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747"
Jul 14 11:14:24.770959 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.770694    2563 scope.go:117] "RemoveContainer" containerID="85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747"
Jul 14 11:14:24.771195 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:24.771175    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747\": container with ID starting with 85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747 not found: ID does not exist" containerID="85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747"
Jul 14 11:14:24.771277 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.771207    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747"} err="failed to get container status \"85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747\": rpc error: code = NotFound desc = could not find container \"85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747\": container with ID starting with 85e3fd31d3e2fb3ca34c445f60da1a4f9e093b4195b672da298eda0373212747 not found: ID does not exist"
Jul 14 11:14:24.781985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.781963    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"]
Jul 14 11:14:24.788722 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:24.788703    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b7d5q"]
Jul 14 11:14:25.826060 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:25.826027    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa8f654-ac13-4f83-bada-2faf03155f1c" path="/var/lib/kubelet/pods/dfa8f654-ac13-4f83-bada-2faf03155f1c/volumes"
Jul 14 11:14:40.098113 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.098086    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"]
Jul 14 11:14:40.098628 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.098451    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa8f654-ac13-4f83-bada-2faf03155f1c" containerName="manager"
Jul 14 11:14:40.098628 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.098462    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa8f654-ac13-4f83-bada-2faf03155f1c" containerName="manager"
Jul 14 11:14:40.098628 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.098508    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa8f654-ac13-4f83-bada-2faf03155f1c" containerName="manager"
Jul 14 11:14:40.103014 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.102992    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.105254 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.105236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9gnx7\""
Jul 14 11:14:40.112946 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.112925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"]
Jul 14 11:14:40.126036 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.126011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:40.281994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.281945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvvz\" (UniqueName: \"kubernetes.io/projected/60afec14-4153-48ac-83e0-7d1cf921bab5-kube-api-access-7cvvz\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60afec14-4153-48ac-83e0-7d1cf921bab5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.282358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.282252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383591 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383591 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60afec14-4153-48ac-83e0-7d1cf921bab5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383591 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvvz\" (UniqueName: \"kubernetes.io/projected/60afec14-4153-48ac-83e0-7d1cf921bab5-kube-api-access-7cvvz\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.383842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.383780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.384169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.384017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.384169 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.384038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.384276 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.384247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.384341 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.384272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.384422 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.384403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60afec14-4153-48ac-83e0-7d1cf921bab5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.386012 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.385985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.386225 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.386206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.392334 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.392307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvvz\" (UniqueName: \"kubernetes.io/projected/60afec14-4153-48ac-83e0-7d1cf921bab5-kube-api-access-7cvvz\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.392497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.392475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60afec14-4153-48ac-83e0-7d1cf921bab5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-kdhgg\" (UID: \"60afec14-4153-48ac-83e0-7d1cf921bab5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.414605 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.414580    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:40.564025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.564003    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"]
Jul 14 11:14:40.565824 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:14:40.565795    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60afec14_4153_48ac_83e0_7d1cf921bab5.slice/crio-c2a64e72627aa30a11a64e4bd67bdb88aedf0340e72aa83260e16394b2609338 WatchSource:0}: Error finding container c2a64e72627aa30a11a64e4bd67bdb88aedf0340e72aa83260e16394b2609338: Status 404 returned error can't find the container with id c2a64e72627aa30a11a64e4bd67bdb88aedf0340e72aa83260e16394b2609338
Jul 14 11:14:40.567777 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.567744    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:14:40.567873 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.567818    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:14:40.567873 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.567845    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:14:40.569515 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.569492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:40.832788 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.832754    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" event={"ID":"60afec14-4153-48ac-83e0-7d1cf921bab5","Type":"ContainerStarted","Data":"3e27323e0ac60da841cc279c80a0365c13b8a4cf5a9de02b7754cb2528b2fbdf"}
Jul 14 11:14:40.833008 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.832806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" event={"ID":"60afec14-4153-48ac-83e0-7d1cf921bab5","Type":"ContainerStarted","Data":"c2a64e72627aa30a11a64e4bd67bdb88aedf0340e72aa83260e16394b2609338"}
Jul 14 11:14:40.857937 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:40.857896    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" podStartSLOduration=0.857881827 podStartE2EDuration="857.881827ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:14:40.856034654 +0000 UTC m=+485.438573681" watchObservedRunningTime="2026-07-14 11:14:40.857881827 +0000 UTC m=+485.440420831"
Jul 14 11:14:41.415625 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:41.415591    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:41.420303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:41.420280    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:41.837354 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:41.837325    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:41.838263 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:41.838245    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:14:41.957886 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:41.957856    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:42.043645 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.043614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:42.143831 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.143754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:42.391641 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.391603    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:14:42.395285 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.395225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.398511 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.398489    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hr6wg\""
Jul 14 11:14:42.398616 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.398489    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 11:14:42.407076 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.407055    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:14:42.440809 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.440777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:14:42.503162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.503134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpnwt\" (UniqueName: \"kubernetes.io/projected/7307c388-ffa8-4a5b-86ed-1bcd809226d7-kube-api-access-rpnwt\") pod \"limitador-limitador-54d6d988d5-fx27v\" (UID: \"7307c388-ffa8-4a5b-86ed-1bcd809226d7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.503162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.503163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7307c388-ffa8-4a5b-86ed-1bcd809226d7-config-file\") pod \"limitador-limitador-54d6d988d5-fx27v\" (UID: \"7307c388-ffa8-4a5b-86ed-1bcd809226d7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.604607 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.604575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpnwt\" (UniqueName: \"kubernetes.io/projected/7307c388-ffa8-4a5b-86ed-1bcd809226d7-kube-api-access-rpnwt\") pod \"limitador-limitador-54d6d988d5-fx27v\" (UID: \"7307c388-ffa8-4a5b-86ed-1bcd809226d7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.604754 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.604613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7307c388-ffa8-4a5b-86ed-1bcd809226d7-config-file\") pod \"limitador-limitador-54d6d988d5-fx27v\" (UID: \"7307c388-ffa8-4a5b-86ed-1bcd809226d7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.605272 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.605255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7307c388-ffa8-4a5b-86ed-1bcd809226d7-config-file\") pod \"limitador-limitador-54d6d988d5-fx27v\" (UID: \"7307c388-ffa8-4a5b-86ed-1bcd809226d7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.616588 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.616560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpnwt\" (UniqueName: \"kubernetes.io/projected/7307c388-ffa8-4a5b-86ed-1bcd809226d7-kube-api-access-rpnwt\") pod \"limitador-limitador-54d6d988d5-fx27v\" (UID: \"7307c388-ffa8-4a5b-86ed-1bcd809226d7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.705815 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.705749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:42.717811 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.717785    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-f5ql6"]
Jul 14 11:14:42.721533 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.721514    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:42.724328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.724307    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4mzdq\""
Jul 14 11:14:42.730573 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.730548    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f5ql6"]
Jul 14 11:14:42.754918 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.754883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:42.836340 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.836312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:42.846240 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.846155    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:14:42.849000 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:14:42.848974    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7307c388_ffa8_4a5b_86ed_1bcd809226d7.slice/crio-7b0244ae2bacd8e74db5580dc1eb8e5a783f95af4f95e20d9b54c75de41d535b WatchSource:0}: Error finding container 7b0244ae2bacd8e74db5580dc1eb8e5a783f95af4f95e20d9b54c75de41d535b: Status 404 returned error can't find the container with id 7b0244ae2bacd8e74db5580dc1eb8e5a783f95af4f95e20d9b54c75de41d535b
Jul 14 11:14:42.908477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:42.908449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpqzv\" (UniqueName: \"kubernetes.io/projected/76f2e12d-c1ab-4e9e-a422-06885dc292cf-kube-api-access-lpqzv\") pod \"authorino-58cd955675-f5ql6\" (UID: \"76f2e12d-c1ab-4e9e-a422-06885dc292cf\") " pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:43.009870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:43.009810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpqzv\" (UniqueName: \"kubernetes.io/projected/76f2e12d-c1ab-4e9e-a422-06885dc292cf-kube-api-access-lpqzv\") pod \"authorino-58cd955675-f5ql6\" (UID: \"76f2e12d-c1ab-4e9e-a422-06885dc292cf\") " pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:43.020180 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:43.020152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpqzv\" (UniqueName: \"kubernetes.io/projected/76f2e12d-c1ab-4e9e-a422-06885dc292cf-kube-api-access-lpqzv\") pod \"authorino-58cd955675-f5ql6\" (UID: \"76f2e12d-c1ab-4e9e-a422-06885dc292cf\") " pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:43.043126 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:43.043106    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:43.167751 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:43.167721    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f5ql6"]
Jul 14 11:14:43.170937 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:14:43.170906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76f2e12d_c1ab_4e9e_a422_06885dc292cf.slice/crio-aa0b1902e01c1bdaa65be7898bf7460ee64adb48fcb2e7c866bb3bad601058be WatchSource:0}: Error finding container aa0b1902e01c1bdaa65be7898bf7460ee64adb48fcb2e7c866bb3bad601058be: Status 404 returned error can't find the container with id aa0b1902e01c1bdaa65be7898bf7460ee64adb48fcb2e7c866bb3bad601058be
Jul 14 11:14:43.849476 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:43.849429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f5ql6" event={"ID":"76f2e12d-c1ab-4e9e-a422-06885dc292cf","Type":"ContainerStarted","Data":"aa0b1902e01c1bdaa65be7898bf7460ee64adb48fcb2e7c866bb3bad601058be"}
Jul 14 11:14:43.852578 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:43.852551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v" event={"ID":"7307c388-ffa8-4a5b-86ed-1bcd809226d7","Type":"ContainerStarted","Data":"7b0244ae2bacd8e74db5580dc1eb8e5a783f95af4f95e20d9b54c75de41d535b"}
Jul 14 11:14:47.313487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:47.313451    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f5ql6"]
Jul 14 11:14:47.875800 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:47.875763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v" event={"ID":"7307c388-ffa8-4a5b-86ed-1bcd809226d7","Type":"ContainerStarted","Data":"074788a78b0c4a9e0e4f133f20bba4fa26a640d4545ced879845d78ce4740eb5"}
Jul 14 11:14:47.875986 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:47.875830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:14:47.877223 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:47.877193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f5ql6" event={"ID":"76f2e12d-c1ab-4e9e-a422-06885dc292cf","Type":"ContainerStarted","Data":"e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3"}
Jul 14 11:14:47.895965 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:47.895921    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v" podStartSLOduration=1.6687325739999999 podStartE2EDuration="5.895908157s" totalImagesPullingTime="4.227175583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:14:47.893759346 +0000 UTC m=+492.476298395" watchObservedRunningTime="2026-07-14 11:14:47.895908157 +0000 UTC m=+492.478447162"
Jul 14 11:14:47.914826 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:47.914761    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-f5ql6" podStartSLOduration=1.962131979 podStartE2EDuration="5.914748816s" totalImagesPullingTime="3.952616837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:14:47.912452093 +0000 UTC m=+492.494991108" watchObservedRunningTime="2026-07-14 11:14:47.914748816 +0000 UTC m=+492.497287821"
Jul 14 11:14:48.882270 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:48.882233    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-f5ql6" podUID="76f2e12d-c1ab-4e9e-a422-06885dc292cf" containerName="authorino" containerID="cri-o://e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3" gracePeriod=30
Jul 14 11:14:49.124881 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.124857    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:49.161424 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.161368    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpqzv\" (UniqueName: \"kubernetes.io/projected/76f2e12d-c1ab-4e9e-a422-06885dc292cf-kube-api-access-lpqzv\") pod \"76f2e12d-c1ab-4e9e-a422-06885dc292cf\" (UID: \"76f2e12d-c1ab-4e9e-a422-06885dc292cf\") "
Jul 14 11:14:49.163248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.163220    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f2e12d-c1ab-4e9e-a422-06885dc292cf-kube-api-access-lpqzv" (OuterVolumeSpecName: "kube-api-access-lpqzv") pod "76f2e12d-c1ab-4e9e-a422-06885dc292cf" (UID: "76f2e12d-c1ab-4e9e-a422-06885dc292cf"). InnerVolumeSpecName "kube-api-access-lpqzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:14:49.262171 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.262132    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpqzv\" (UniqueName: \"kubernetes.io/projected/76f2e12d-c1ab-4e9e-a422-06885dc292cf-kube-api-access-lpqzv\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:14:49.887733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.887698    2563 generic.go:356] "Generic (PLEG): container finished" podID="76f2e12d-c1ab-4e9e-a422-06885dc292cf" containerID="e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3" exitCode=0
Jul 14 11:14:49.888102 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.887742    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f5ql6"
Jul 14 11:14:49.888102 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.887795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f5ql6" event={"ID":"76f2e12d-c1ab-4e9e-a422-06885dc292cf","Type":"ContainerDied","Data":"e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3"}
Jul 14 11:14:49.888102 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.887830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f5ql6" event={"ID":"76f2e12d-c1ab-4e9e-a422-06885dc292cf","Type":"ContainerDied","Data":"aa0b1902e01c1bdaa65be7898bf7460ee64adb48fcb2e7c866bb3bad601058be"}
Jul 14 11:14:49.888102 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.887846    2563 scope.go:117] "RemoveContainer" containerID="e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3"
Jul 14 11:14:49.895892 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.895876    2563 scope.go:117] "RemoveContainer" containerID="e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3"
Jul 14 11:14:49.896133 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:14:49.896113    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3\": container with ID starting with e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3 not found: ID does not exist" containerID="e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3"
Jul 14 11:14:49.896203 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.896139    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3"} err="failed to get container status \"e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3\": rpc error: code = NotFound desc = could not find container \"e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3\": container with ID starting with e0e4b7360ca028f4c8e67a16e0b83405e34de6b1cb4d039a203ad2a2e42cacf3 not found: ID does not exist"
Jul 14 11:14:49.908923 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.908901    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f5ql6"]
Jul 14 11:14:49.910559 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:49.910541    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-f5ql6"]
Jul 14 11:14:51.825906 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:51.825873    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f2e12d-c1ab-4e9e-a422-06885dc292cf" path="/var/lib/kubelet/pods/76f2e12d-c1ab-4e9e-a422-06885dc292cf/volumes"
Jul 14 11:14:53.664570 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:53.664528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:53.765216 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:53.765187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:58.395292 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:58.395250    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:58.461584 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:58.461553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:14:58.883614 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:14:58.883583    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-fx27v"
Jul 14 11:15:03.229560 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:03.229515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:03.328060 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:03.328024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:03.714123 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:03.714092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:16.178397 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.178361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:16.390035 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.389992    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:16.435895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.435823    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gm9wh"]
Jul 14 11:15:16.436257 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.436240    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f2e12d-c1ab-4e9e-a422-06885dc292cf" containerName="authorino"
Jul 14 11:15:16.436347 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.436259    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f2e12d-c1ab-4e9e-a422-06885dc292cf" containerName="authorino"
Jul 14 11:15:16.436401 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.436356    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f2e12d-c1ab-4e9e-a422-06885dc292cf" containerName="authorino"
Jul 14 11:15:16.443449 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.443427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:16.445967 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.445943    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4mzdq\""
Jul 14 11:15:16.448035 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.448009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gm9wh"]
Jul 14 11:15:16.583249 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.583216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw6mn\" (UniqueName: \"kubernetes.io/projected/45c48900-fa88-4c2d-ae3f-6343b29f3a0c-kube-api-access-kw6mn\") pod \"authorino-5d4b67c446-gm9wh\" (UID: \"45c48900-fa88-4c2d-ae3f-6343b29f3a0c\") " pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:16.681967 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.681939    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gm9wh"]
Jul 14 11:15:16.682186 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:15:16.682167    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kw6mn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-gm9wh" podUID="45c48900-fa88-4c2d-ae3f-6343b29f3a0c"
Jul 14 11:15:16.684301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.684279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw6mn\" (UniqueName: \"kubernetes.io/projected/45c48900-fa88-4c2d-ae3f-6343b29f3a0c-kube-api-access-kw6mn\") pod \"authorino-5d4b67c446-gm9wh\" (UID: \"45c48900-fa88-4c2d-ae3f-6343b29f3a0c\") " pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:16.692281 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:16.692219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw6mn\" (UniqueName: \"kubernetes.io/projected/45c48900-fa88-4c2d-ae3f-6343b29f3a0c-kube-api-access-kw6mn\") pod \"authorino-5d4b67c446-gm9wh\" (UID: \"45c48900-fa88-4c2d-ae3f-6343b29f3a0c\") " pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:17.005642 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:17.005567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:17.058286 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:17.058259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:17.188561 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:17.188532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw6mn\" (UniqueName: \"kubernetes.io/projected/45c48900-fa88-4c2d-ae3f-6343b29f3a0c-kube-api-access-kw6mn\") pod \"45c48900-fa88-4c2d-ae3f-6343b29f3a0c\" (UID: \"45c48900-fa88-4c2d-ae3f-6343b29f3a0c\") "
Jul 14 11:15:17.190595 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:17.190563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45c48900-fa88-4c2d-ae3f-6343b29f3a0c-kube-api-access-kw6mn" (OuterVolumeSpecName: "kube-api-access-kw6mn") pod "45c48900-fa88-4c2d-ae3f-6343b29f3a0c" (UID: "45c48900-fa88-4c2d-ae3f-6343b29f3a0c"). InnerVolumeSpecName "kube-api-access-kw6mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:15:17.289475 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:17.289440    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw6mn\" (UniqueName: \"kubernetes.io/projected/45c48900-fa88-4c2d-ae3f-6343b29f3a0c-kube-api-access-kw6mn\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:15:18.010641 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:18.010612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gm9wh"
Jul 14 11:15:18.038428 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:18.038400    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gm9wh"]
Jul 14 11:15:18.040279 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:18.040258    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gm9wh"]
Jul 14 11:15:19.826114 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:19.826080    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45c48900-fa88-4c2d-ae3f-6343b29f3a0c" path="/var/lib/kubelet/pods/45c48900-fa88-4c2d-ae3f-6343b29f3a0c/volumes"
Jul 14 11:15:23.634289 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:23.634256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:23.735209 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:23.735178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:23.838071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:23.838037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:23.999459 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:23.999364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:27.400179 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:27.400148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:27.523022 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:27.522985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:47.922472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:47.922440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:47.992188 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:47.992156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:48.644569 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:48.644535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:48.712923 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:48.712892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:15:59.963952 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:15:59.963913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:00.065243 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:00.065211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:00.481047 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:00.481009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:00.747445 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:00.747365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:01.310674 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:01.310613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:01.461117 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:01.461078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:01.681949 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:01.681871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:16:09.265586 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:09.265559    2563 generic.go:356] "Generic (PLEG): container finished" podID="60afec14-4153-48ac-83e0-7d1cf921bab5" containerID="3e27323e0ac60da841cc279c80a0365c13b8a4cf5a9de02b7754cb2528b2fbdf" exitCode=0
Jul 14 11:16:09.265950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:09.265609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" event={"ID":"60afec14-4153-48ac-83e0-7d1cf921bab5","Type":"ContainerDied","Data":"3e27323e0ac60da841cc279c80a0365c13b8a4cf5a9de02b7754cb2528b2fbdf"}
Jul 14 11:16:09.265994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:09.265972    2563 scope.go:117] "RemoveContainer" containerID="3e27323e0ac60da841cc279c80a0365c13b8a4cf5a9de02b7754cb2528b2fbdf"
Jul 14 11:16:09.267115 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:09.267085    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:16:09.267196 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:09.267173    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:16:09.267234 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:09.267220    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 14 11:16:10.271733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:10.271684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" event={"ID":"60afec14-4153-48ac-83e0-7d1cf921bab5","Type":"ContainerStarted","Data":"3c9e380edf3d19b0e26b0d21387af33ccb4be9dcb757de504f78f2ed095f55d4"}
Jul 14 11:16:10.313569 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:10.313543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:10.415668 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:10.415616    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:16:10.416572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:10.416547    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kdhgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 14 11:16:10.416706 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:10.416593    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" podUID="60afec14-4153-48ac-83e0-7d1cf921bab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 14 11:16:10.448062 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:10.448025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:11.415156 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:11.415118    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kdhgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 14 11:16:11.415548 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:11.415185    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" podUID="60afec14-4153-48ac-83e0-7d1cf921bab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 14 11:16:11.838084 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:11.838055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:16:12.415241 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:12.415209    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kdhgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 14 11:16:12.415646 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:12.415277    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" podUID="60afec14-4153-48ac-83e0-7d1cf921bab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 14 11:16:13.414968 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:13.414933    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kdhgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 14 11:16:13.415142 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:13.415002    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg" podUID="60afec14-4153-48ac-83e0-7d1cf921bab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 14 11:16:14.419149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:14.419118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:16:14.420105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:14.420086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kdhgg"
Jul 14 11:16:17.463591 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:17.463540    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:27.767219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:27.767177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:28.035505 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:28.035469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:16:35.766015 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:35.765986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:16:35.775975 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:35.775953    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:16:35.976378 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:35.976339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:36.836414 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:36.836377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:16:57.049220 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:57.049174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:16:58.138928 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:16:58.138849    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:17:04.674829 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:04.674789    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:17:06.241900 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:06.241864    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:17:09.062757 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:09.062720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:17:10.456017 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:10.455982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:17:19.660365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:19.660334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:17:21.735144 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:21.735107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:17:23.444935 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.444892    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"]
Jul 14 11:17:23.448444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.448425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.452753 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.452720    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 11:17:23.452875 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.452759    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 14 11:17:23.452875 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.452797    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 11:17:23.452875 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.452762    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ln5z2\""
Jul 14 11:17:23.467681 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.467646    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"]
Jul 14 11:17:23.505094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.505057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:17:23.565371 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.565344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.565523 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.565379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.565523 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.565400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.565523 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.565416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.565670 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.565526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d56d27a-bff6-4450-82a5-799ad65fd08d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.565670 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.565574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvncw\" (UniqueName: \"kubernetes.io/projected/7d56d27a-bff6-4450-82a5-799ad65fd08d-kube-api-access-zvncw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.666623 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.666595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d56d27a-bff6-4450-82a5-799ad65fd08d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.666804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.666637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvncw\" (UniqueName: \"kubernetes.io/projected/7d56d27a-bff6-4450-82a5-799ad65fd08d-kube-api-access-zvncw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.666804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.666685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.666804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.666705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.666804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.666722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.666804 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.666739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.667177 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.667153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.667243 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.667162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.667243 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.667218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.668991 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.668967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d56d27a-bff6-4450-82a5-799ad65fd08d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.669098 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.669072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d56d27a-bff6-4450-82a5-799ad65fd08d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.675214 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.675188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvncw\" (UniqueName: \"kubernetes.io/projected/7d56d27a-bff6-4450-82a5-799ad65fd08d-kube-api-access-zvncw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r\" (UID: \"7d56d27a-bff6-4450-82a5-799ad65fd08d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.758708 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.758610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:23.905439 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:17:23.905412    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d56d27a_bff6_4450_82a5_799ad65fd08d.slice/crio-8a5ec19ef8c4282e260be7c4e4fe93fa56966cb48ca241aaf4b6eea432ae52a6 WatchSource:0}: Error finding container 8a5ec19ef8c4282e260be7c4e4fe93fa56966cb48ca241aaf4b6eea432ae52a6: Status 404 returned error can't find the container with id 8a5ec19ef8c4282e260be7c4e4fe93fa56966cb48ca241aaf4b6eea432ae52a6
Jul 14 11:17:23.906725 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.906701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"]
Jul 14 11:17:23.906946 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:23.906925    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:17:24.616728 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:24.616690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r" event={"ID":"7d56d27a-bff6-4450-82a5-799ad65fd08d","Type":"ContainerStarted","Data":"8a5ec19ef8c4282e260be7c4e4fe93fa56966cb48ca241aaf4b6eea432ae52a6"}
Jul 14 11:17:25.622634 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:25.622601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r" event={"ID":"7d56d27a-bff6-4450-82a5-799ad65fd08d","Type":"ContainerStarted","Data":"0a73ad3f745f9a9ac36bd32de1019d4d42ce0da2b7b55db652876998e0091f00"}
Jul 14 11:17:25.647149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:25.647102    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r" podStartSLOduration=1.616978705 podStartE2EDuration="2.647086149s" totalImagesPullingTime="1.030107444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:17:25.6466486 +0000 UTC m=+650.229187605" watchObservedRunningTime="2026-07-14 11:17:25.647086149 +0000 UTC m=+650.229625154"
Jul 14 11:17:31.145146 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:31.145111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:17:32.711486 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:32.711450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:17:33.759132 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:33.759096    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:33.759518 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:33.759179    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:33.771569 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:33.771546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:17:34.676535 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:17:34.676506    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r"
Jul 14 11:18:25.818686 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:25.818641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:18:38.954450 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.954370    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-854559bc69-zn6gc"]
Jul 14 11:18:38.957558 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.957538    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:38.961601 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.961580    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 14 11:18:38.961730 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.961581    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4mzdq\""
Jul 14 11:18:38.970736 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.970712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-854559bc69-zn6gc"]
Jul 14 11:18:38.996878 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.996851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbr6w\" (UniqueName: \"kubernetes.io/projected/babebe6f-47b8-42a6-a170-421fde3c0e0b-kube-api-access-wbr6w\") pod \"authorino-854559bc69-zn6gc\" (UID: \"babebe6f-47b8-42a6-a170-421fde3c0e0b\") " pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:38.997021 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:38.996920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/babebe6f-47b8-42a6-a170-421fde3c0e0b-tls-cert\") pod \"authorino-854559bc69-zn6gc\" (UID: \"babebe6f-47b8-42a6-a170-421fde3c0e0b\") " pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:39.098107 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.098068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbr6w\" (UniqueName: \"kubernetes.io/projected/babebe6f-47b8-42a6-a170-421fde3c0e0b-kube-api-access-wbr6w\") pod \"authorino-854559bc69-zn6gc\" (UID: \"babebe6f-47b8-42a6-a170-421fde3c0e0b\") " pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:39.098283 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.098157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/babebe6f-47b8-42a6-a170-421fde3c0e0b-tls-cert\") pod \"authorino-854559bc69-zn6gc\" (UID: \"babebe6f-47b8-42a6-a170-421fde3c0e0b\") " pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:39.100706 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.100685    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/babebe6f-47b8-42a6-a170-421fde3c0e0b-tls-cert\") pod \"authorino-854559bc69-zn6gc\" (UID: \"babebe6f-47b8-42a6-a170-421fde3c0e0b\") " pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:39.114774 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.114749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbr6w\" (UniqueName: \"kubernetes.io/projected/babebe6f-47b8-42a6-a170-421fde3c0e0b-kube-api-access-wbr6w\") pod \"authorino-854559bc69-zn6gc\" (UID: \"babebe6f-47b8-42a6-a170-421fde3c0e0b\") " pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:39.266355 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.266278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854559bc69-zn6gc"
Jul 14 11:18:39.416861 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.416829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-854559bc69-zn6gc"]
Jul 14 11:18:39.419908 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:18:39.419876    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbabebe6f_47b8_42a6_a170_421fde3c0e0b.slice/crio-92f94aad25c12803240fd3c5c8705017425ff096de31180ef3edb2277c124c84 WatchSource:0}: Error finding container 92f94aad25c12803240fd3c5c8705017425ff096de31180ef3edb2277c124c84: Status 404 returned error can't find the container with id 92f94aad25c12803240fd3c5c8705017425ff096de31180ef3edb2277c124c84
Jul 14 11:18:39.468435 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.468375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:18:39.587008 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.586976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:18:39.959205 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:39.959182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-854559bc69-zn6gc" event={"ID":"babebe6f-47b8-42a6-a170-421fde3c0e0b","Type":"ContainerStarted","Data":"92f94aad25c12803240fd3c5c8705017425ff096de31180ef3edb2277c124c84"}
Jul 14 11:18:40.700125 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:40.700088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:18:40.816393 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:40.816356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:18:40.966261 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:40.966165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-854559bc69-zn6gc" event={"ID":"babebe6f-47b8-42a6-a170-421fde3c0e0b","Type":"ContainerStarted","Data":"ed79c1ca2774b9352e258769de4762bdb03acfab41163f56fd5ef06509bc347c"}
Jul 14 11:18:41.002043 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:18:41.001977    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-854559bc69-zn6gc" podStartSLOduration=2.523692149 podStartE2EDuration="3.001954432s" totalImagesPullingTime="478.262283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:40.999416497 +0000 UTC m=+725.581955503" watchObservedRunningTime="2026-07-14 11:18:41.001954432 +0000 UTC m=+725.584493439"
Jul 14 11:19:10.546080 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:10.546036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:19:21.171108 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:21.171073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:19:31.760780 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:31.760748    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:19:40.842436 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:40.842402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:19:51.633070 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:51.633034    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:19:54.076764 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:54.076695    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:19:55.408984 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:19:55.408952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:18.940922 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:18.940885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:20:31.915514 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:31.915472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:32.028001 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:32.027970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:32.248726 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:32.248630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:20:35.274368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:35.274332    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:35.373279 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:35.373244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:35.550891 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:35.550811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:41.549617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:41.549586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:41.653760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:41.653724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:41.760023 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:41.759988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:41.924266 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:41.924177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:47.958400 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:47.958361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:20:48.062920 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:20:48.062887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:21:04.045022 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:04.044985    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:21:11.932834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:11.932801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:21:22.558597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:22.558559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:21:30.939552 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:30.939462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:21:35.795911 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:35.795880    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:21:35.802425 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:35.802396    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:21:41.843342 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:41.843310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:21:54.554687 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:21:54.554639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:22:05.242197 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:22:05.242160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:22:15.335643 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:22:15.335602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:22:36.730097 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:22:36.730059    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:22:48.744943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:22:48.744908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:23:13.146513 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:23:13.146472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:23:23.737488 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:23:23.737448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:23:38.734060 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:23:38.734019    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:23:49.538468 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:23:49.538433    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:23:59.079574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:23:59.079534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:23:59.444619 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:23:59.444540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:24:08.737996 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:08.737964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:24:18.248001 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:18.247963    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:24:46.314841 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:46.314750    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"]
Jul 14 11:24:46.315392 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:46.314917    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs" podUID="6e8f61dd-d325-4f38-8bc7-55926cc48a55" containerName="manager" containerID="cri-o://11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb" gracePeriod=10
Jul 14 11:24:47.963129 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:47.963104    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:24:48.010362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.010324    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e8f61dd-d325-4f38-8bc7-55926cc48a55-extensions-socket-volume\") pod \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") "
Jul 14 11:24:48.010698 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.010374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8968d\" (UniqueName: \"kubernetes.io/projected/6e8f61dd-d325-4f38-8bc7-55926cc48a55-kube-api-access-8968d\") pod \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\" (UID: \"6e8f61dd-d325-4f38-8bc7-55926cc48a55\") "
Jul 14 11:24:48.010831 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.010735    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e8f61dd-d325-4f38-8bc7-55926cc48a55-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6e8f61dd-d325-4f38-8bc7-55926cc48a55" (UID: "6e8f61dd-d325-4f38-8bc7-55926cc48a55"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:24:48.012416 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.012392    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e8f61dd-d325-4f38-8bc7-55926cc48a55-kube-api-access-8968d" (OuterVolumeSpecName: "kube-api-access-8968d") pod "6e8f61dd-d325-4f38-8bc7-55926cc48a55" (UID: "6e8f61dd-d325-4f38-8bc7-55926cc48a55"). InnerVolumeSpecName "kube-api-access-8968d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:24:48.111617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.111585    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e8f61dd-d325-4f38-8bc7-55926cc48a55-extensions-socket-volume\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:24:48.111617 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.111611    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8968d\" (UniqueName: \"kubernetes.io/projected/6e8f61dd-d325-4f38-8bc7-55926cc48a55-kube-api-access-8968d\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:24:48.728678 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.728626    2563 generic.go:356] "Generic (PLEG): container finished" podID="6e8f61dd-d325-4f38-8bc7-55926cc48a55" containerID="11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb" exitCode=0
Jul 14 11:24:48.728846 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.728693    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"
Jul 14 11:24:48.728846 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.728698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs" event={"ID":"6e8f61dd-d325-4f38-8bc7-55926cc48a55","Type":"ContainerDied","Data":"11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb"}
Jul 14 11:24:48.728846 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.728757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs" event={"ID":"6e8f61dd-d325-4f38-8bc7-55926cc48a55","Type":"ContainerDied","Data":"22a0168443b3f46aa66d41c19265d8f9bd197bac68fe2b04c9b08853cfe84cc1"}
Jul 14 11:24:48.728846 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.728781    2563 scope.go:117] "RemoveContainer" containerID="11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb"
Jul 14 11:24:48.738502 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.738485    2563 scope.go:117] "RemoveContainer" containerID="11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb"
Jul 14 11:24:48.738765 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:24:48.738745    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb\": container with ID starting with 11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb not found: ID does not exist" containerID="11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb"
Jul 14 11:24:48.738835 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.738772    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb"} err="failed to get container status \"11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb\": rpc error: code = NotFound desc = could not find container \"11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb\": container with ID starting with 11c17759b56e2bc4797a1f867026962c4f43c5ef640247bf9da8e885bbadc2cb not found: ID does not exist"
Jul 14 11:24:48.751692 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.751670    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"]
Jul 14 11:24:48.755553 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:48.755531    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8btxs"]
Jul 14 11:24:49.826043 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:24:49.826009    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e8f61dd-d325-4f38-8bc7-55926cc48a55" path="/var/lib/kubelet/pods/6e8f61dd-d325-4f38-8bc7-55926cc48a55/volumes"
Jul 14 11:25:05.305305 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:05.305264    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:25:13.687618 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:13.687584    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:25:13.791723 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:13.791692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:25:47.662336 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.662300    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"]
Jul 14 11:25:47.662790 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.662637    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e8f61dd-d325-4f38-8bc7-55926cc48a55" containerName="manager"
Jul 14 11:25:47.662790 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.662648    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e8f61dd-d325-4f38-8bc7-55926cc48a55" containerName="manager"
Jul 14 11:25:47.662790 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.662730    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e8f61dd-d325-4f38-8bc7-55926cc48a55" containerName="manager"
Jul 14 11:25:47.666974 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.666958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.670182 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.670161    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dzxhj\""
Jul 14 11:25:47.715671 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.715628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"]
Jul 14 11:25:47.803100 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.803062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmpmz\" (UniqueName: \"kubernetes.io/projected/de9bebf1-1874-40a0-9e1c-c6065aaababe-kube-api-access-dmpmz\") pod \"kuadrant-operator-controller-manager-86878c769f-zlmdb\" (UID: \"de9bebf1-1874-40a0-9e1c-c6065aaababe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.803269 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.803131    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de9bebf1-1874-40a0-9e1c-c6065aaababe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zlmdb\" (UID: \"de9bebf1-1874-40a0-9e1c-c6065aaababe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.904202 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.904168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmpmz\" (UniqueName: \"kubernetes.io/projected/de9bebf1-1874-40a0-9e1c-c6065aaababe-kube-api-access-dmpmz\") pod \"kuadrant-operator-controller-manager-86878c769f-zlmdb\" (UID: \"de9bebf1-1874-40a0-9e1c-c6065aaababe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.904387 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.904238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de9bebf1-1874-40a0-9e1c-c6065aaababe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zlmdb\" (UID: \"de9bebf1-1874-40a0-9e1c-c6065aaababe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.904615 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.904595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de9bebf1-1874-40a0-9e1c-c6065aaababe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zlmdb\" (UID: \"de9bebf1-1874-40a0-9e1c-c6065aaababe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.914445 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.914375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmpmz\" (UniqueName: \"kubernetes.io/projected/de9bebf1-1874-40a0-9e1c-c6065aaababe-kube-api-access-dmpmz\") pod \"kuadrant-operator-controller-manager-86878c769f-zlmdb\" (UID: \"de9bebf1-1874-40a0-9e1c-c6065aaababe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:47.976619 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:47.976586    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:48.124219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:48.124127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"]
Jul 14 11:25:48.127084 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:25:48.127062    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9bebf1_1874_40a0_9e1c_c6065aaababe.slice/crio-59a7528480a950daf1e9768c0357f5c640a5c78e8bc5f1914e9472e0ae49b428 WatchSource:0}: Error finding container 59a7528480a950daf1e9768c0357f5c640a5c78e8bc5f1914e9472e0ae49b428: Status 404 returned error can't find the container with id 59a7528480a950daf1e9768c0357f5c640a5c78e8bc5f1914e9472e0ae49b428
Jul 14 11:25:49.011437 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:49.011397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb" event={"ID":"de9bebf1-1874-40a0-9e1c-c6065aaababe","Type":"ContainerStarted","Data":"41b8e49ef6f86714ce424fabf12b14438e939df82ab7158d6b6db4db6c1de693"}
Jul 14 11:25:49.011893 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:49.011586    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:25:49.011893 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:25:49.011613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb" event={"ID":"de9bebf1-1874-40a0-9e1c-c6065aaababe","Type":"ContainerStarted","Data":"59a7528480a950daf1e9768c0357f5c640a5c78e8bc5f1914e9472e0ae49b428"}
Jul 14 11:26:00.018620 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:00.018583    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb"
Jul 14 11:26:00.040533 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:00.040478    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlmdb" podStartSLOduration=13.040462935 podStartE2EDuration="13.040462935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:25:49.043138507 +0000 UTC m=+1153.625677509" watchObservedRunningTime="2026-07-14 11:26:00.040462935 +0000 UTC m=+1164.623001939"
Jul 14 11:26:10.240017 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:10.239983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:26:22.348772 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:22.348737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:26:30.644237 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:30.644198    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:26:35.833214 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:35.833187    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:26:35.835356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:35.835337    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:26:44.876087 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:44.876049    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:26:58.057790 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:26:58.057741    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:27:10.527413 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:10.527372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:27:19.933613 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:19.933579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:27:28.460195 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:28.460108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:27:31.917861 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:31.917817    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:27:40.333954 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:40.333916    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:27:43.004829 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:43.004791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:27:51.846889 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:27:51.846855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:28:01.529483 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:28:01.529442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:28:17.790353 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:28:17.790316    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:28:26.952451 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:28:26.952414    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:28:37.637882 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:28:37.637847    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:28:49.245077 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:28:49.245039    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:28:57.332742 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:28:57.332711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:29:02.736590 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:02.736551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:29:04.019232 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:04.019191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:29:12.639422 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:12.639386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:29:24.846966 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:24.846916    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:29:33.841795 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:33.841764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:29:47.845689 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:47.845641    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:29:57.550777 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:29:57.550745    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:30:10.961255 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:30:10.961216    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:30:20.132762 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:30:20.132728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:30:30.151294 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:30:30.151222    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:30:42.634057 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:30:42.634023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:30:53.348852 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:30:53.348816    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:31:06.045296 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:06.045256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:31:09.021625 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:09.021593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:31:19.103498 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:19.103466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:31:31.755578 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:31.755542    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:31:35.861313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:35.861285    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:31:35.861732 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:35.861713    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:31:40.964099 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:40.964062    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:31:44.023243 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:44.023211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:31:52.212231 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:52.212168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:31:54.958492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:54.958457    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:31:59.830293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:31:59.830176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:05.210301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:05.210256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:06.845948 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:06.845915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:32:08.334227 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:08.334197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:08.472310 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:08.472260    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:08.804279 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:08.804245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:15.903953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:15.903912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:16.005287 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:16.005257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:16.133676 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:16.133621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:16.203030 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:16.202906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:17.442612 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:17.442581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:17.686733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:17.686699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:17.862795 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:17.862763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:18.408085 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:18.408020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:18.508438 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:18.508398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:19.714513 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:19.714479    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:24.137782 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:24.137745    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:32:57.939262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:57.939210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:32:58.048476 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:32:58.048449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:32.995954 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:32.995867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:36.033649 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:36.033611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:33:37.505688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:37.505593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:44.064506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:44.064464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:44.166380 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:44.166332    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:47.835870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:47.835840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:48.168843 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:48.168747    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:48.293144 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:48.293113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:48.407842 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:48.407812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:49.610689 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:49.610638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:49.744428 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:49.744397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:49.973739 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:49.973627    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:50.455356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:50.455306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:50.548308 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:50.548270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:54.425453 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:54.425422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:54.527448 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:54.527415    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:33:56.744690 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:56.744642    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:33:57.822760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:33:57.822719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:34:54.317964 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:34:54.317932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:34:55.496809 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:34:55.496773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:34:57.135406 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:34:57.135370    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:35:00.487352 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:00.487320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:00.693328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:00.693297    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:00.853357 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:00.853323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:00.965585 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:00.965557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:01.971320 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:01.971285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:02.119196 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:02.119169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:02.233380 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:02.233285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:02.355153 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:02.355125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:02.922566 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:02.922524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:03.042226 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:03.042194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:04.331038 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:04.331007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:04.347038 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:04.347010    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:35:17.599202 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:17.599164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:18.279950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:18.279913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:18.406031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:18.405999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:20.335302 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:20.335265    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:35:22.193036 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:22.193004    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:28.079147 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:28.079086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:28.179641 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:28.179605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:33.268191 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:33.268149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:33.569107 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:33.569075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:34.675459 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:34.675426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:34.773684 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:34.772497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:34.844217 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:34.844184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:35.159037 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:35.159006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:35.347352 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:35.347311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:35.447820 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:35.447737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:37.234825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:37.234789    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:35:45.233719 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:45.233690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:45.333669 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:45.333623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:35:48.757794 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:35:48.757757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:35.888571 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:35.888541    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:36:35.890404 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:35.890385    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:36:37.076143 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:37.076109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:37.131018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:37.130976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:39.632514 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:39.632487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:42.626500 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:42.626463    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:36:44.278759 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:44.278724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:50.016945 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:50.016912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:50.117773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:50.117739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:54.429816 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:54.429780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:55.051458 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:55.051427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:55.187252 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:55.187221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:55.341350 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:55.341258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:36:56.773101 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:36:56.773068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:01.115627 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:01.115596    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:02.743620 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:02.743585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:02.863559 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:02.863524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:02.957922 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:02.957891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:03.079464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:03.079425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:03.493487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:03.493400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:29.490091 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:29.490050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:35.340408 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:35.340374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:35.406908 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:35.406878    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:35.498988 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:35.498956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:35.597272 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:35.597190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:51.517380 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:51.517348    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:57.460395 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:57.460361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:57.573799 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:57.573766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:37:57.664105 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:37:57.664067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:08.232896 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:08.232864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:09.062303 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:09.062271    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:15.376047 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:15.376014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:15.479221 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:15.479179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:15.575074 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:15.575044    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:15.683076 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:15.682996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:16.514353 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:16.514320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:16.612206 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:16.612159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:16.713499 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:16.713466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:18.038190 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:18.038157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:38:18.519444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:18.519344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:24.378593 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:24.378560    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:24.475509 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:24.475480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:25.081941 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:25.081903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:25.535067 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:25.535038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:38:31.013367 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:31.013326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:31.114310 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:31.114273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:31.296153 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:31.296116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:31.649338 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:31.649234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:31.957300 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:31.957206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:34.539424 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:34.539382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:38:34.671943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:34.671877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:34.770479 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:34.770426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:34.944664 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:34.944554    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:35.224328 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:35.224232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:35.584114 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:35.584082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:35.692498 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:35.692431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:39.128531 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:39.128497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:39.820429 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:39.820312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:39.911819 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:39.911786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:40.270232 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:40.270141    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:44.834420 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:44.834382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:38:46.331229 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:46.331194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:46.388031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:46.388001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:51.717607 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:51.717542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:38:57.237370 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:38:57.237330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:00.037603 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:00.037564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:39:01.804318 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:01.804284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:07.647512 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:07.647477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:07.750827 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:07.750790    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:12.514716 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:12.514685    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:12.749021 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:12.748989    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:12.882133 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:12.882098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:12.982125 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:12.982081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:13.120696 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:13.120640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:18.842627 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:18.842586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:18.951901 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:18.951850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:19.043470 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:19.043418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:19.226527 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:19.226447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:25.381300 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:25.381269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:25.480626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:25.480593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:32.360697 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:32.360573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:32.692526 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:32.692430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:33.663786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:33.663750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:35.643167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:35.643130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:35.753974 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:35.753936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:35.857270 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:35.857222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:36.412172 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:36.412140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:36.523456 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:36.523421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:40.090083 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:40.090047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:41.242506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:41.242471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:39:50.431792 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.431752    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"]
Jul 14 11:39:50.435517 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.435494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.438845 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.438821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c5ce9ebd\"/\"e2e-lifecycle-model-c5ce9ebd-kserve-self-signed-certs\""
Jul 14 11:39:50.438959 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.438847    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c5ce9ebd\"/\"default-dockercfg-kk6qf\""
Jul 14 11:39:50.438959 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.438821    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c5ce9ebd\"/\"kube-root-ca.crt\""
Jul 14 11:39:50.438959 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.438820    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c5ce9ebd\"/\"openshift-service-ca.crt\""
Jul 14 11:39:50.445511 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.445490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"]
Jul 14 11:39:50.490143 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.490115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-dshm\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.490291 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.490149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-model-cache\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.490291 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.490218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-tmp-dir\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.490291 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.490253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b03076c-412c-4bc5-a342-43b849ced322-tls-certs\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.490418 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.490299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-home\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.490418 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.490347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxpr4\" (UniqueName: \"kubernetes.io/projected/5b03076c-412c-4bc5-a342-43b849ced322-kube-api-access-cxpr4\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.540789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.540755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:39:50.590927 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.590894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-home\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591107 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.590938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxpr4\" (UniqueName: \"kubernetes.io/projected/5b03076c-412c-4bc5-a342-43b849ced322-kube-api-access-cxpr4\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591107 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-dshm\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591241 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-model-cache\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591241 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-tmp-dir\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591241 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b03076c-412c-4bc5-a342-43b849ced322-tls-certs\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591397 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-home\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-model-cache\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.591520 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.591460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-tmp-dir\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.593187 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.593162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-dshm\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.593574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.593558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b03076c-412c-4bc5-a342-43b849ced322-tls-certs\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.599127 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.599104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxpr4\" (UniqueName: \"kubernetes.io/projected/5b03076c-412c-4bc5-a342-43b849ced322-kube-api-access-cxpr4\") pod \"e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") " pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.745309 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.745218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:39:50.877361 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:50.877331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"]
Jul 14 11:39:50.881842 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:39:50.881815    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b03076c_412c_4bc5_a342_43b849ced322.slice/crio-80fa09885cb033549cf2320a3b9ac3132ffba3bf6b8fc61a01afdf14c7cab23a WatchSource:0}: Error finding container 80fa09885cb033549cf2320a3b9ac3132ffba3bf6b8fc61a01afdf14c7cab23a: Status 404 returned error can't find the container with id 80fa09885cb033549cf2320a3b9ac3132ffba3bf6b8fc61a01afdf14c7cab23a
Jul 14 11:39:51.027262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:51.027228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n" event={"ID":"5b03076c-412c-4bc5-a342-43b849ced322","Type":"ContainerStarted","Data":"a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716"}
Jul 14 11:39:51.027412 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:51.027271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n" event={"ID":"5b03076c-412c-4bc5-a342-43b849ced322","Type":"ContainerStarted","Data":"80fa09885cb033549cf2320a3b9ac3132ffba3bf6b8fc61a01afdf14c7cab23a"}
Jul 14 11:39:51.049247 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:51.049200    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n" podStartSLOduration=1.049185334 podStartE2EDuration="1.049185334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:39:51.046959779 +0000 UTC m=+1995.629498794" watchObservedRunningTime="2026-07-14 11:39:51.049185334 +0000 UTC m=+1995.631724339"
Jul 14 11:39:53.533167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:39:53.533132    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:40:00.746292 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:00.746257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:40:00.746292 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:00.746298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:40:00.758883 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:00.758859    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:40:01.087688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:01.087644    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:40:19.228025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:19.227991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:20.976983 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:20.976945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:25.537175 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:25.537143    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:40:29.244631 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:29.244602    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:29.345631 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:29.345591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:31.928764 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:31.928733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:39.288176 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:39.288118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:42.648024 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:42.647987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:40:43.831248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:43.831211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:40:48.801675 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:48.801620    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"]
Jul 14 11:40:48.847607 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:40:48.847579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:03.828958 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:03.828858    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n" podUID="5b03076c-412c-4bc5-a342-43b849ced322" containerName="main" containerID="cri-o://a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716" gracePeriod=45
Jul 14 11:41:04.082436 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.082381    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:41:04.225492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225457    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-dshm\") pod \"5b03076c-412c-4bc5-a342-43b849ced322\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") "
Jul 14 11:41:04.225492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-home\") pod \"5b03076c-412c-4bc5-a342-43b849ced322\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") "
Jul 14 11:41:04.225773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-model-cache\") pod \"5b03076c-412c-4bc5-a342-43b849ced322\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") "
Jul 14 11:41:04.225773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225561    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-tmp-dir\") pod \"5b03076c-412c-4bc5-a342-43b849ced322\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") "
Jul 14 11:41:04.225773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225597    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxpr4\" (UniqueName: \"kubernetes.io/projected/5b03076c-412c-4bc5-a342-43b849ced322-kube-api-access-cxpr4\") pod \"5b03076c-412c-4bc5-a342-43b849ced322\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") "
Jul 14 11:41:04.225773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b03076c-412c-4bc5-a342-43b849ced322-tls-certs\") pod \"5b03076c-412c-4bc5-a342-43b849ced322\" (UID: \"5b03076c-412c-4bc5-a342-43b849ced322\") "
Jul 14 11:41:04.225994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225868    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-home" (OuterVolumeSpecName: "home") pod "5b03076c-412c-4bc5-a342-43b849ced322" (UID: "5b03076c-412c-4bc5-a342-43b849ced322"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:41:04.225994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225886    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b03076c-412c-4bc5-a342-43b849ced322" (UID: "5b03076c-412c-4bc5-a342-43b849ced322"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:41:04.225994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225902    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b03076c-412c-4bc5-a342-43b849ced322" (UID: "5b03076c-412c-4bc5-a342-43b849ced322"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:41:04.225994 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.225942    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:41:04.227924 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.227889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-dshm" (OuterVolumeSpecName: "dshm") pod "5b03076c-412c-4bc5-a342-43b849ced322" (UID: "5b03076c-412c-4bc5-a342-43b849ced322"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:41:04.227924 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.227906    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b03076c-412c-4bc5-a342-43b849ced322-kube-api-access-cxpr4" (OuterVolumeSpecName: "kube-api-access-cxpr4") pod "5b03076c-412c-4bc5-a342-43b849ced322" (UID: "5b03076c-412c-4bc5-a342-43b849ced322"). InnerVolumeSpecName "kube-api-access-cxpr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:41:04.228085 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.227920    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b03076c-412c-4bc5-a342-43b849ced322-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b03076c-412c-4bc5-a342-43b849ced322" (UID: "5b03076c-412c-4bc5-a342-43b849ced322"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:41:04.326692 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.326638    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:41:04.326852 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.326696    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxpr4\" (UniqueName: \"kubernetes.io/projected/5b03076c-412c-4bc5-a342-43b849ced322-kube-api-access-cxpr4\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:41:04.326852 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.326714    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b03076c-412c-4bc5-a342-43b849ced322-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:41:04.326852 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.326726    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:41:04.326852 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.326735    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b03076c-412c-4bc5-a342-43b849ced322-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:41:04.388832 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.388746    2563 generic.go:356] "Generic (PLEG): container finished" podID="5b03076c-412c-4bc5-a342-43b849ced322" containerID="a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716" exitCode=0
Jul 14 11:41:04.388832 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.388782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n" event={"ID":"5b03076c-412c-4bc5-a342-43b849ced322","Type":"ContainerDied","Data":"a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716"}
Jul 14 11:41:04.388832 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.388811    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"
Jul 14 11:41:04.388832 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.388809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n" event={"ID":"5b03076c-412c-4bc5-a342-43b849ced322","Type":"ContainerDied","Data":"80fa09885cb033549cf2320a3b9ac3132ffba3bf6b8fc61a01afdf14c7cab23a"}
Jul 14 11:41:04.389117 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.388839    2563 scope.go:117] "RemoveContainer" containerID="a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716"
Jul 14 11:41:04.397711 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.397693    2563 scope.go:117] "RemoveContainer" containerID="a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716"
Jul 14 11:41:04.397947 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:41:04.397928    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716\": container with ID starting with a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716 not found: ID does not exist" containerID="a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716"
Jul 14 11:41:04.398019 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.397953    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716"} err="failed to get container status \"a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716\": rpc error: code = NotFound desc = could not find container \"a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716\": container with ID starting with a726a8de29386c9b1f866f242eb76e5c0c12a7c22b8620684c3c5e1ccd78d716 not found: ID does not exist"
Jul 14 11:41:04.409774 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.409752    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"]
Jul 14 11:41:04.416444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:04.416425    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c5ce9ebd/e2e-lifecycle-model-c5ce9ebd-kserve-654d74968c-wz97n"]
Jul 14 11:41:05.827281 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:05.827247    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b03076c-412c-4bc5-a342-43b849ced322" path="/var/lib/kubelet/pods/5b03076c-412c-4bc5-a342-43b849ced322/volumes"
Jul 14 11:41:20.370628 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:20.370596    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:20.428950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:20.428916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:25.719187 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:25.719150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:26.086957 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:26.082436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:31.855492 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:31.855455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:35.914551 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:35.914521    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:41:35.917747 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:35.917724    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:41:46.089615 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:46.089576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:46.189075 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:46.189046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:56.227766 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:56.227726    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:56.463984 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:56.463937    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:57.561618 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:57.561584    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:57.658931 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:57.658897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:57.757307 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:57.757281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:58.558362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:58.558326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:58.644310 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:58.644273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:41:58.745585 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:41:58.745557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:00.191777 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:00.191737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:00.437910 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:00.437874    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:42:06.036480 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:06.036447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:06.108792 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:06.108756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:06.799185 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:06.799137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:12.878877 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:12.878842    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:12.980179 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:12.980124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:13.077223 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:13.077186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:14.338019 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:14.337984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:42:23.638158 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:23.638119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:24.006480 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:24.006398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:45.006314 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:45.006224    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:46.186084 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:46.186051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:50.854477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:50.854441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:50.954632 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:50.954599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:51.240220 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:51.240129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:51.554622 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:51.554583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:52.113829 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:52.113793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:54.486256 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:54.486216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:54.586780 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:54.586722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:54.664614 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:54.664578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:55.085577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:55.085545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:55.183386 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:55.183354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:55.284397 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:55.284367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:59.023331 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:59.023277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:59.376476 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:59.376390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:42:59.728213 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:42:59.728106    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:00.068168 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:00.068111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:02.434975 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:02.434933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:43:02.588867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:02.588834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:02.692514 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:02.692427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:02.850824 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:02.850778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:02.951042 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:02.950934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:03.338833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:03.338801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:03.441542 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:03.441508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:07.122573 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:07.122535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:13.629167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:13.629122    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:43:14.232472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.232433    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"]
Jul 14 11:43:14.232825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.232809    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b03076c-412c-4bc5-a342-43b849ced322" containerName="main"
Jul 14 11:43:14.232892 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.232827    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b03076c-412c-4bc5-a342-43b849ced322" containerName="main"
Jul 14 11:43:14.232929 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.232894    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b03076c-412c-4bc5-a342-43b849ced322" containerName="main"
Jul 14 11:43:14.235887 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.235870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.238441 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.238418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1b0afa\"/\"kube-root-ca.crt\""
Jul 14 11:43:14.239597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.239331    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1b0afa\"/\"default-dockercfg-l6cjw\""
Jul 14 11:43:14.239597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.239361    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1b0afa\"/\"openshift-service-ca.crt\""
Jul 14 11:43:14.239597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.239395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1b0afa\"/\"test-model-1b0afa-kserve-self-signed-certs\""
Jul 14 11:43:14.244947 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.244925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"]
Jul 14 11:43:14.268406 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.268382    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:14.296514 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.296494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-dshm\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.296610 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.296525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz5k2\" (UniqueName: \"kubernetes.io/projected/334c9b5d-35d8-47be-993f-d369842b575b-kube-api-access-tz5k2\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.296610 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.296554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-model-cache\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.296610 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.296598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-home\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.296758 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.296634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/334c9b5d-35d8-47be-993f-d369842b575b-tls-certs\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.296758 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.296705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-tmp-dir\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398142 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz5k2\" (UniqueName: \"kubernetes.io/projected/334c9b5d-35d8-47be-993f-d369842b575b-kube-api-access-tz5k2\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-model-cache\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-home\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/334c9b5d-35d8-47be-993f-d369842b575b-tls-certs\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398470 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-tmp-dir\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398586 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-dshm\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398676 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398574    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-home\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398676 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-model-cache\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.398786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.398733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-tmp-dir\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.400699 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.400674    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-dshm\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.400949 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.400930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/334c9b5d-35d8-47be-993f-d369842b575b-tls-certs\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.406137 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.406114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz5k2\" (UniqueName: \"kubernetes.io/projected/334c9b5d-35d8-47be-993f-d369842b575b-kube-api-access-tz5k2\") pod \"test-model-1b0afa-kserve-95cc59675-w7pww\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") " pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.546816 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.546789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:14.683378 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:14.683355    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"]
Jul 14 11:43:14.686195 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:43:14.686167    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod334c9b5d_35d8_47be_993f_d369842b575b.slice/crio-6a5dc3dca9dae5bc768a62e811a85dc8b9db870a73065b73b62a30a28ba8d605 WatchSource:0}: Error finding container 6a5dc3dca9dae5bc768a62e811a85dc8b9db870a73065b73b62a30a28ba8d605: Status 404 returned error can't find the container with id 6a5dc3dca9dae5bc768a62e811a85dc8b9db870a73065b73b62a30a28ba8d605
Jul 14 11:43:15.002041 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:15.001962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww" event={"ID":"334c9b5d-35d8-47be-993f-d369842b575b","Type":"ContainerStarted","Data":"a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6"}
Jul 14 11:43:15.002041 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:15.002002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww" event={"ID":"334c9b5d-35d8-47be-993f-d369842b575b","Type":"ContainerStarted","Data":"6a5dc3dca9dae5bc768a62e811a85dc8b9db870a73065b73b62a30a28ba8d605"}
Jul 14 11:43:15.020373 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:15.020321    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww" podStartSLOduration=1.020306803 podStartE2EDuration="1.020306803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:43:15.018829918 +0000 UTC m=+2199.601368923" watchObservedRunningTime="2026-07-14 11:43:15.020306803 +0000 UTC m=+2199.602845808"
Jul 14 11:43:15.980755 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:15.980720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:16.080680 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:16.080630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:23.534826 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:23.534785    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:43:24.137445 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:24.137394    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:24.242985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:24.242951    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:24.547431 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:24.547401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:24.547870 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:24.547692    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:24.560763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:24.560739    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:25.060996 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:25.060969    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:43:41.355809 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.355771    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"]
Jul 14 11:43:41.359102 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.359075    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.362728 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.362708    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f85160\"/\"default-dockercfg-7czqx\""
Jul 14 11:43:41.362820 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.362710    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f85160\"/\"test-model-f85160-kserve-self-signed-certs\""
Jul 14 11:43:41.364260 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.364241    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f85160\"/\"openshift-service-ca.crt\""
Jul 14 11:43:41.364346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.364328    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f85160\"/\"kube-root-ca.crt\""
Jul 14 11:43:41.373902 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.373884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"]
Jul 14 11:43:41.408198 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.408174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:43:41.534920 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.534887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-model-cache\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.535072 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.534967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-home\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.535072 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.534987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-dshm\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.535072 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.535006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9nsm\" (UniqueName: \"kubernetes.io/projected/440f86f7-f129-40ab-a5c3-91c1d1553870-kube-api-access-t9nsm\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.535189 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.535094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-tmp-dir\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.535189 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.535129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/440f86f7-f129-40ab-a5c3-91c1d1553870-tls-certs\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.636553 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-model-cache\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.636553 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-home\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.636769 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-dshm\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.636769 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9nsm\" (UniqueName: \"kubernetes.io/projected/440f86f7-f129-40ab-a5c3-91c1d1553870-kube-api-access-t9nsm\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.636769 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-tmp-dir\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.636926 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/440f86f7-f129-40ab-a5c3-91c1d1553870-tls-certs\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.637001 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-home\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.637069 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.636999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-model-cache\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.637130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.637098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-tmp-dir\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.638708 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.638683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-dshm\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.639084 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.639068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/440f86f7-f129-40ab-a5c3-91c1d1553870-tls-certs\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.645214 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.645190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9nsm\" (UniqueName: \"kubernetes.io/projected/440f86f7-f129-40ab-a5c3-91c1d1553870-kube-api-access-t9nsm\") pod \"test-model-f85160-kserve-6955fdb677-lzzfz\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") " pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.669150 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.669130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:41.799594 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:41.799571    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"]
Jul 14 11:43:41.801756 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:43:41.801729    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod440f86f7_f129_40ab_a5c3_91c1d1553870.slice/crio-d79194f417f46a72e53fca6a73939802760db906163c266d14cb22ab09f20dd9 WatchSource:0}: Error finding container d79194f417f46a72e53fca6a73939802760db906163c266d14cb22ab09f20dd9: Status 404 returned error can't find the container with id d79194f417f46a72e53fca6a73939802760db906163c266d14cb22ab09f20dd9
Jul 14 11:43:42.136688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:42.136644    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz" event={"ID":"440f86f7-f129-40ab-a5c3-91c1d1553870","Type":"ContainerStarted","Data":"e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239"}
Jul 14 11:43:42.136688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:42.136694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz" event={"ID":"440f86f7-f129-40ab-a5c3-91c1d1553870","Type":"ContainerStarted","Data":"d79194f417f46a72e53fca6a73939802760db906163c266d14cb22ab09f20dd9"}
Jul 14 11:43:42.180704 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:42.180643    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz" podStartSLOduration=1.180626411 podStartE2EDuration="1.180626411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:43:42.180490446 +0000 UTC m=+2226.763029450" watchObservedRunningTime="2026-07-14 11:43:42.180626411 +0000 UTC m=+2226.763165416"
Jul 14 11:43:45.137216 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:45.137188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:43:51.669822 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:51.669771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:51.670422 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:51.669841    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:51.682728 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:51.682694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:43:52.198711 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:43:52.198679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:44:00.676902 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:00.676689    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"]
Jul 14 11:44:00.698674 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:00.698623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:01.657069 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:01.657037    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"]
Jul 14 11:44:01.686997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:01.686962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:02.318357 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:02.318324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:02.416178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:02.416134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:04.268761 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:04.268725    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:44:12.260754 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:12.260717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:15.698226 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:15.698156    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww" podUID="334c9b5d-35d8-47be-993f-d369842b575b" containerName="main" containerID="cri-o://a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6" gracePeriod=45
Jul 14 11:44:15.952196 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:15.952132    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:44:16.030215 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030180    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-model-cache\") pod \"334c9b5d-35d8-47be-993f-d369842b575b\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") "
Jul 14 11:44:16.030363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-tmp-dir\") pod \"334c9b5d-35d8-47be-993f-d369842b575b\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") "
Jul 14 11:44:16.030363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-dshm\") pod \"334c9b5d-35d8-47be-993f-d369842b575b\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") "
Jul 14 11:44:16.030363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030307    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-home\") pod \"334c9b5d-35d8-47be-993f-d369842b575b\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") "
Jul 14 11:44:16.030363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz5k2\" (UniqueName: \"kubernetes.io/projected/334c9b5d-35d8-47be-993f-d369842b575b-kube-api-access-tz5k2\") pod \"334c9b5d-35d8-47be-993f-d369842b575b\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") "
Jul 14 11:44:16.030572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/334c9b5d-35d8-47be-993f-d369842b575b-tls-certs\") pod \"334c9b5d-35d8-47be-993f-d369842b575b\" (UID: \"334c9b5d-35d8-47be-993f-d369842b575b\") "
Jul 14 11:44:16.030572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030499    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-model-cache" (OuterVolumeSpecName: "model-cache") pod "334c9b5d-35d8-47be-993f-d369842b575b" (UID: "334c9b5d-35d8-47be-993f-d369842b575b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:16.030572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "334c9b5d-35d8-47be-993f-d369842b575b" (UID: "334c9b5d-35d8-47be-993f-d369842b575b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:16.030770 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030620    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-home" (OuterVolumeSpecName: "home") pod "334c9b5d-35d8-47be-993f-d369842b575b" (UID: "334c9b5d-35d8-47be-993f-d369842b575b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:16.030770 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030632    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:16.030770 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.030682    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:16.032680 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.032637    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-dshm" (OuterVolumeSpecName: "dshm") pod "334c9b5d-35d8-47be-993f-d369842b575b" (UID: "334c9b5d-35d8-47be-993f-d369842b575b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:16.032776 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.032687    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/334c9b5d-35d8-47be-993f-d369842b575b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "334c9b5d-35d8-47be-993f-d369842b575b" (UID: "334c9b5d-35d8-47be-993f-d369842b575b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:44:16.032776 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.032731    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/334c9b5d-35d8-47be-993f-d369842b575b-kube-api-access-tz5k2" (OuterVolumeSpecName: "kube-api-access-tz5k2") pod "334c9b5d-35d8-47be-993f-d369842b575b" (UID: "334c9b5d-35d8-47be-993f-d369842b575b"). InnerVolumeSpecName "kube-api-access-tz5k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:44:16.131638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.131597    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:16.131638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.131631    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/334c9b5d-35d8-47be-993f-d369842b575b-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:16.131638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.131642    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz5k2\" (UniqueName: \"kubernetes.io/projected/334c9b5d-35d8-47be-993f-d369842b575b-kube-api-access-tz5k2\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:16.132148 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.131681    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/334c9b5d-35d8-47be-993f-d369842b575b-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:16.315157 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.315118    2563 generic.go:356] "Generic (PLEG): container finished" podID="334c9b5d-35d8-47be-993f-d369842b575b" containerID="a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6" exitCode=0
Jul 14 11:44:16.315371 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.315174    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"
Jul 14 11:44:16.315371 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.315204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww" event={"ID":"334c9b5d-35d8-47be-993f-d369842b575b","Type":"ContainerDied","Data":"a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6"}
Jul 14 11:44:16.315371 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.315251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww" event={"ID":"334c9b5d-35d8-47be-993f-d369842b575b","Type":"ContainerDied","Data":"6a5dc3dca9dae5bc768a62e811a85dc8b9db870a73065b73b62a30a28ba8d605"}
Jul 14 11:44:16.315371 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.315270    2563 scope.go:117] "RemoveContainer" containerID="a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6"
Jul 14 11:44:16.324438 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.324423    2563 scope.go:117] "RemoveContainer" containerID="a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6"
Jul 14 11:44:16.324728 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:44:16.324711    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6\": container with ID starting with a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6 not found: ID does not exist" containerID="a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6"
Jul 14 11:44:16.324796 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.324735    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6"} err="failed to get container status \"a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6\": rpc error: code = NotFound desc = could not find container \"a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6\": container with ID starting with a8080efc8e28fb4c9524b2cde40c32d4fce972ae0a33f9be884c640dcedce3c6 not found: ID does not exist"
Jul 14 11:44:16.337139 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.337110    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"]
Jul 14 11:44:16.341121 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.341094    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1b0afa/test-model-1b0afa-kserve-95cc59675-w7pww"]
Jul 14 11:44:16.682897 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.682793    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz" podUID="440f86f7-f129-40ab-a5c3-91c1d1553870" containerName="main" containerID="cri-o://e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239" gracePeriod=45
Jul 14 11:44:16.888357 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.888322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:16.933574 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:16.933508    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:44:17.038860 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.038824    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9nsm\" (UniqueName: \"kubernetes.io/projected/440f86f7-f129-40ab-a5c3-91c1d1553870-kube-api-access-t9nsm\") pod \"440f86f7-f129-40ab-a5c3-91c1d1553870\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") "
Jul 14 11:44:17.039068 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.038891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/440f86f7-f129-40ab-a5c3-91c1d1553870-tls-certs\") pod \"440f86f7-f129-40ab-a5c3-91c1d1553870\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") "
Jul 14 11:44:17.039068 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.038924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-model-cache\") pod \"440f86f7-f129-40ab-a5c3-91c1d1553870\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") "
Jul 14 11:44:17.039068 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039022    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-tmp-dir\") pod \"440f86f7-f129-40ab-a5c3-91c1d1553870\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") "
Jul 14 11:44:17.039251 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039108    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-home\") pod \"440f86f7-f129-40ab-a5c3-91c1d1553870\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") "
Jul 14 11:44:17.039251 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039149    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-model-cache" (OuterVolumeSpecName: "model-cache") pod "440f86f7-f129-40ab-a5c3-91c1d1553870" (UID: "440f86f7-f129-40ab-a5c3-91c1d1553870"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:17.039251 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-dshm\") pod \"440f86f7-f129-40ab-a5c3-91c1d1553870\" (UID: \"440f86f7-f129-40ab-a5c3-91c1d1553870\") "
Jul 14 11:44:17.039408 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039301    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "440f86f7-f129-40ab-a5c3-91c1d1553870" (UID: "440f86f7-f129-40ab-a5c3-91c1d1553870"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:17.039408 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039334    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-home" (OuterVolumeSpecName: "home") pod "440f86f7-f129-40ab-a5c3-91c1d1553870" (UID: "440f86f7-f129-40ab-a5c3-91c1d1553870"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:17.039510 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039490    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:17.039571 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039512    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:17.039571 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.039524    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:17.041218 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.041184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440f86f7-f129-40ab-a5c3-91c1d1553870-kube-api-access-t9nsm" (OuterVolumeSpecName: "kube-api-access-t9nsm") pod "440f86f7-f129-40ab-a5c3-91c1d1553870" (UID: "440f86f7-f129-40ab-a5c3-91c1d1553870"). InnerVolumeSpecName "kube-api-access-t9nsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:44:17.041331 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.041239    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/440f86f7-f129-40ab-a5c3-91c1d1553870-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "440f86f7-f129-40ab-a5c3-91c1d1553870" (UID: "440f86f7-f129-40ab-a5c3-91c1d1553870"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:44:17.041331 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.041249    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-dshm" (OuterVolumeSpecName: "dshm") pod "440f86f7-f129-40ab-a5c3-91c1d1553870" (UID: "440f86f7-f129-40ab-a5c3-91c1d1553870"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:44:17.140789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.140734    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/440f86f7-f129-40ab-a5c3-91c1d1553870-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:17.140789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.140780    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/440f86f7-f129-40ab-a5c3-91c1d1553870-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:17.140789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.140793    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9nsm\" (UniqueName: \"kubernetes.io/projected/440f86f7-f129-40ab-a5c3-91c1d1553870-kube-api-access-t9nsm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:44:17.321111 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.321073    2563 generic.go:356] "Generic (PLEG): container finished" podID="440f86f7-f129-40ab-a5c3-91c1d1553870" containerID="e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239" exitCode=0
Jul 14 11:44:17.321111 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.321117    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"
Jul 14 11:44:17.321350 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.321160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz" event={"ID":"440f86f7-f129-40ab-a5c3-91c1d1553870","Type":"ContainerDied","Data":"e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239"}
Jul 14 11:44:17.321350 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.321207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz" event={"ID":"440f86f7-f129-40ab-a5c3-91c1d1553870","Type":"ContainerDied","Data":"d79194f417f46a72e53fca6a73939802760db906163c266d14cb22ab09f20dd9"}
Jul 14 11:44:17.321350 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.321238    2563 scope.go:117] "RemoveContainer" containerID="e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239"
Jul 14 11:44:17.335803 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.335783    2563 scope.go:117] "RemoveContainer" containerID="e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239"
Jul 14 11:44:17.336083 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:44:17.336059    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239\": container with ID starting with e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239 not found: ID does not exist" containerID="e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239"
Jul 14 11:44:17.336183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.336089    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239"} err="failed to get container status \"e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239\": rpc error: code = NotFound desc = could not find container \"e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239\": container with ID starting with e5f97206333aecc75ef48b45de4eef6da56f80caf2d476423ffd708288382239 not found: ID does not exist"
Jul 14 11:44:17.355965 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.355935    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"]
Jul 14 11:44:17.369797 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.369770    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f85160/test-model-f85160-kserve-6955fdb677-lzzfz"]
Jul 14 11:44:17.826736 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.826702    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="334c9b5d-35d8-47be-993f-d369842b575b" path="/var/lib/kubelet/pods/334c9b5d-35d8-47be-993f-d369842b575b/volumes"
Jul 14 11:44:17.827222 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:17.827202    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="440f86f7-f129-40ab-a5c3-91c1d1553870" path="/var/lib/kubelet/pods/440f86f7-f129-40ab-a5c3-91c1d1553870/volumes"
Jul 14 11:44:19.233167 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:19.233129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:44:22.797495 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:22.797462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:22.898649 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:22.898609    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:27.610239 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:27.610207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:27.864748 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:27.864617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:28.498438 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:28.498406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:28.601082 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:28.601048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:38.451613 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:38.451581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:41.533137 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:41.533098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:44:43.080896 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:43.080832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:49.033455 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:49.033421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:49.109374 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:49.109315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:53.801089 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:53.801052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:54.037170 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:54.037139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:54.406688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:54.406641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:54.730078 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:54.729953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:55.077333 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:55.077304    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:58.243368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:58.243338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:58.335208 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:58.335172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:58.434384 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:58.434347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:58.788361 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:58.788325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:59.015680 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:59.015637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:44:59.114576 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:44:59.114483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:02.736467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:02.736430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:02.829003 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:02.828973    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:45:03.226466 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:03.226429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:03.557444 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:03.557410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:03.893627 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:03.893533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:06.286571 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:06.286524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:06.380219 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:06.380183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:06.563376 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:06.563283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:06.951313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:06.951226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:07.086867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:07.086832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:07.187647 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:07.187607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:10.824308 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:10.824270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:11.477352 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:11.477318    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:14.765111 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:14.765071    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:45:19.605773 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:19.605733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:19.703807 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:19.703767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:20.450318 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:20.450284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:20.593090 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:20.593062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:21.465604 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:21.465571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:22.630748 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:22.630704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:29.087162 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:29.087083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:29.198374 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:29.198344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:29.608829 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:29.608793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:32.939363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:32.939329    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:45:34.146764 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:34.146711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:40.079621 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:40.079574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:40.180262 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:40.180230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:44.819045 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:44.818993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:45.066160 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:45.066129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:55.126154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:55.126121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:55.823426 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:55.823393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:45:58.873875 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:45:58.873844    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:46:00.365782 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:00.365735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:06.322431 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:06.322399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:06.421730 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:06.421693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:11.099210 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:11.099170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:11.340366 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:11.340326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:11.747213 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:11.747177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:12.292980 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:12.292944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:12.604441 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:12.604357    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:15.732241 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:15.732208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:15.830102 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:15.830049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:15.914884 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:15.914844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:16.214001 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:16.213965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:16.670982 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:16.670950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:16.769394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:16.769364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:19.854152 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:19.854117    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:46:20.288622 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:20.288589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:20.687178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:20.687082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:21.010411 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:21.010320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:21.117282 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:21.117248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:23.758502 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:23.758470    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:23.896337 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:23.896306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:24.173607 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:24.173570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:24.519895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:24.519775    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:24.695263 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:24.695226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:24.817549 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:24.817502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:28.260369 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:28.260303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:30.943667 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:30.943627    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:46:35.940545 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:35.940516    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:46:35.946071 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:35.946051    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:46:37.570607 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:37.570570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:37.676612 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:37.676553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:45.580672 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.580626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:45.682394 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.682360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:45.940597 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.940506    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"]
Jul 14 11:46:45.940970 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.940950    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="440f86f7-f129-40ab-a5c3-91c1d1553870" containerName="main"
Jul 14 11:46:45.941061 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.940972    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="440f86f7-f129-40ab-a5c3-91c1d1553870" containerName="main"
Jul 14 11:46:45.941061 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.941016    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="334c9b5d-35d8-47be-993f-d369842b575b" containerName="main"
Jul 14 11:46:45.941061 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.941027    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="334c9b5d-35d8-47be-993f-d369842b575b" containerName="main"
Jul 14 11:46:45.941221 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.941127    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="334c9b5d-35d8-47be-993f-d369842b575b" containerName="main"
Jul 14 11:46:45.941221 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.941139    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="440f86f7-f129-40ab-a5c3-91c1d1553870" containerName="main"
Jul 14 11:46:45.944195 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.944173    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:45.947777 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.947754    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6b8edc\"/\"kube-root-ca.crt\""
Jul 14 11:46:45.947895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.947755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b8edc\"/\"auth-test-model-6b8edc-kserve-self-signed-certs\""
Jul 14 11:46:45.947895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.947755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b8edc\"/\"default-dockercfg-pdg4r\""
Jul 14 11:46:45.947895 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.947763    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6b8edc\"/\"openshift-service-ca.crt\""
Jul 14 11:46:45.953725 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:45.953703    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"]
Jul 14 11:46:46.041938 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.041903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:46:46.087199 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.087165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-dshm\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.087362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.087221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snqdj\" (UniqueName: \"kubernetes.io/projected/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-kube-api-access-snqdj\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.087362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.087297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-model-cache\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.087362 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.087334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tls-certs\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.087497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.087361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tmp-dir\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.087497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.087402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-home\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.188544 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.188505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snqdj\" (UniqueName: \"kubernetes.io/projected/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-kube-api-access-snqdj\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.188734 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.188593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-model-cache\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.188734 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.188628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tls-certs\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.188734 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.188681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tmp-dir\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.188734 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.188731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-home\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.188963 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.188784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-dshm\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.189080 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.189053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-model-cache\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.189209 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.189096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tmp-dir\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.189209 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.189144    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-home\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.191018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.190960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-dshm\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.191221 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.191206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tls-certs\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.198904 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.198884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snqdj\" (UniqueName: \"kubernetes.io/projected/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-kube-api-access-snqdj\") pod \"auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") " pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.254227 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.254195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:46.390346 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:46.390325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"]
Jul 14 11:46:46.392723 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:46:46.392697    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ac23fb1_ec62_4e2b_8d43_4288e39eed31.slice/crio-6d288fa6434a20fcc31c647a426a1011bd55927fd6151dd8eb0d024388b29a5d WatchSource:0}: Error finding container 6d288fa6434a20fcc31c647a426a1011bd55927fd6151dd8eb0d024388b29a5d: Status 404 returned error can't find the container with id 6d288fa6434a20fcc31c647a426a1011bd55927fd6151dd8eb0d024388b29a5d
Jul 14 11:46:47.051793 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:47.051752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs" event={"ID":"8ac23fb1-ec62-4e2b-8d43-4288e39eed31","Type":"ContainerStarted","Data":"6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1"}
Jul 14 11:46:47.051793 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:47.051793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs" event={"ID":"8ac23fb1-ec62-4e2b-8d43-4288e39eed31","Type":"ContainerStarted","Data":"6d288fa6434a20fcc31c647a426a1011bd55927fd6151dd8eb0d024388b29a5d"}
Jul 14 11:46:49.442050 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:49.441997    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs" podStartSLOduration=4.441977656 podStartE2EDuration="4.441977656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:47.096076125 +0000 UTC m=+2411.678615141" watchObservedRunningTime="2026-07-14 11:46:49.441977656 +0000 UTC m=+2414.024516754"
Jul 14 11:46:49.443469 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:49.443448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:46:56.254939 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:56.254905    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:56.254939 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:56.254946    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:56.267602 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:56.267573    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:46:57.112741 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:46:57.112713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:47:01.900228 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:01.900153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:47:05.810129 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:05.810090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:47:05.884959 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:05.884928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:47:12.947896 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.947864    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"]
Jul 14 11:47:12.951180 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.951162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:12.953648 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.953625    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"kube-root-ca.crt\""
Jul 14 11:47:12.954549 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.954526    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"openshift-service-ca.crt\""
Jul 14 11:47:12.954649 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.954558    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"default-dockercfg-6ljq7\""
Jul 14 11:47:12.954720 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.954648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"auth-test-model-5c5a7d-kserve-self-signed-certs\""
Jul 14 11:47:12.960626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.960603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"]
Jul 14 11:47:12.973920 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:12.973893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:47:13.027713 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.027677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-model-cache\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.027713 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.027715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzdlg\" (UniqueName: \"kubernetes.io/projected/3c02751e-0864-4b10-97cc-546598f90d68-kube-api-access-fzdlg\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.027950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.027779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-home\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.027950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.027820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-tmp-dir\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.027950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.027872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-dshm\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.027950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.027901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02751e-0864-4b10-97cc-546598f90d68-tls-certs\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.128519 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.128483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-model-cache\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.128519 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.128519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzdlg\" (UniqueName: \"kubernetes.io/projected/3c02751e-0864-4b10-97cc-546598f90d68-kube-api-access-fzdlg\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.128898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.128558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-home\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.128898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.128579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-tmp-dir\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.128898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.128615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-dshm\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.128898 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.128643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02751e-0864-4b10-97cc-546598f90d68-tls-certs\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.129115 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.129000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-model-cache\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.129207 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.129189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-tmp-dir\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.129275 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.129234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-home\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.131174 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.131155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-dshm\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.131390 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.131366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02751e-0864-4b10-97cc-546598f90d68-tls-certs\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.138419 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.138395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzdlg\" (UniqueName: \"kubernetes.io/projected/3c02751e-0864-4b10-97cc-546598f90d68-kube-api-access-fzdlg\") pod \"auth-test-model-5c5a7d-kserve-5844846699-4kd5q\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") " pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.261269 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.261188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:13.392733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:13.392710    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"]
Jul 14 11:47:13.394536 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:47:13.394507    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c02751e_0864_4b10_97cc_546598f90d68.slice/crio-b9219ceaae5feee662afbaa0e4e788ce345548877599111c7f38980c52d97485 WatchSource:0}: Error finding container b9219ceaae5feee662afbaa0e4e788ce345548877599111c7f38980c52d97485: Status 404 returned error can't find the container with id b9219ceaae5feee662afbaa0e4e788ce345548877599111c7f38980c52d97485
Jul 14 11:47:14.184181 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:14.184149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q" event={"ID":"3c02751e-0864-4b10-97cc-546598f90d68","Type":"ContainerStarted","Data":"b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366"}
Jul 14 11:47:14.184545 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:14.184188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q" event={"ID":"3c02751e-0864-4b10-97cc-546598f90d68","Type":"ContainerStarted","Data":"b9219ceaae5feee662afbaa0e4e788ce345548877599111c7f38980c52d97485"}
Jul 14 11:47:14.202067 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:14.202021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q" podStartSLOduration=2.202006876 podStartE2EDuration="2.202006876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:47:14.201104855 +0000 UTC m=+2438.783643859" watchObservedRunningTime="2026-07-14 11:47:14.202006876 +0000 UTC m=+2438.784545880"
Jul 14 11:47:16.555688 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:16.555639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:47:23.261753 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:23.261719    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:23.262150 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:23.261768    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:23.274990 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:23.274966    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:24.247978 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:24.247950    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:47:31.067325 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:31.067283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:47:47.353364 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:47:47.353326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:48:04.040487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:48:04.040451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:48:21.241791 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:48:21.241748    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:48:36.133952 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:48:36.133878    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:48:53.242224 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:48:53.242187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:49:05.322149 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:05.322117    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"]
Jul 14 11:49:05.389457 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:05.389428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:06.312487 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:06.312452    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"]
Jul 14 11:49:06.339233 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:06.339199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:09.035284 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:09.035249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:49:12.349395 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.349333    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"]
Jul 14 11:49:12.353070 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.353047    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.355458 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.355433    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b8edc\"/\"sub-test-model-6b8edc-kserve-self-signed-certs\""
Jul 14 11:49:12.362092 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.362065    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"]
Jul 14 11:49:12.367833 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.367805    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:12.496551 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.496516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6cx5\" (UniqueName: \"kubernetes.io/projected/8be73d31-91eb-49b6-9990-fac5443323aa-kube-api-access-z6cx5\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.496735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.496559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-home\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.496735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.496575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-dshm\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.496735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.496597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-tmp-dir\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.496735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.496612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8be73d31-91eb-49b6-9990-fac5443323aa-tls-certs\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.496735 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.496634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-model-cache\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.527917 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.527885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:12.597798 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.597759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6cx5\" (UniqueName: \"kubernetes.io/projected/8be73d31-91eb-49b6-9990-fac5443323aa-kube-api-access-z6cx5\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.597997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.597813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-home\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.597997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.597842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-dshm\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.597997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.597871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-tmp-dir\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.597997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.597892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8be73d31-91eb-49b6-9990-fac5443323aa-tls-certs\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.598225 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.598114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-model-cache\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.598356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.598333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-home\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.598428 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.598373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-tmp-dir\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.598464 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.598435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-model-cache\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.600119 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.600065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-dshm\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.600363 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.600336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8be73d31-91eb-49b6-9990-fac5443323aa-tls-certs\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.606008 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.605988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6cx5\" (UniqueName: \"kubernetes.io/projected/8be73d31-91eb-49b6-9990-fac5443323aa-kube-api-access-z6cx5\") pod \"sub-test-model-6b8edc-kserve-767f6c8df9-lb66n\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") " pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.663301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.663269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:12.808296 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:12.808271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"]
Jul 14 11:49:12.809867 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:49:12.809832    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8be73d31_91eb_49b6_9990_fac5443323aa.slice/crio-4714493fc756627989e9525a5f2e34cd30165af8a482d0933d606f4f6bc9fafb WatchSource:0}: Error finding container 4714493fc756627989e9525a5f2e34cd30165af8a482d0933d606f4f6bc9fafb: Status 404 returned error can't find the container with id 4714493fc756627989e9525a5f2e34cd30165af8a482d0933d606f4f6bc9fafb
Jul 14 11:49:13.776154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:13.776116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n" event={"ID":"8be73d31-91eb-49b6-9990-fac5443323aa","Type":"ContainerStarted","Data":"6f4c60a8379544f34b8b16a886322b2674f97a14b2a92860014e537e53c50051"}
Jul 14 11:49:13.776154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:13.776161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n" event={"ID":"8be73d31-91eb-49b6-9990-fac5443323aa","Type":"ContainerStarted","Data":"4714493fc756627989e9525a5f2e34cd30165af8a482d0933d606f4f6bc9fafb"}
Jul 14 11:49:13.794961 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:13.794909    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n" podStartSLOduration=1.794893243 podStartE2EDuration="1.794893243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:49:13.793835951 +0000 UTC m=+2558.376374957" watchObservedRunningTime="2026-07-14 11:49:13.794893243 +0000 UTC m=+2558.377432290"
Jul 14 11:49:20.342241 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.342201    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs" podUID="8ac23fb1-ec62-4e2b-8d43-4288e39eed31" containerName="main" containerID="cri-o://6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1" gracePeriod=45
Jul 14 11:49:20.607178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.607123    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:49:20.769693 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769643    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-model-cache\") pod \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") "
Jul 14 11:49:20.769857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tmp-dir\") pod \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") "
Jul 14 11:49:20.769857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769732    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-home\") pod \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") "
Jul 14 11:49:20.769857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-dshm\") pod \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") "
Jul 14 11:49:20.769857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769824    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snqdj\" (UniqueName: \"kubernetes.io/projected/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-kube-api-access-snqdj\") pod \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") "
Jul 14 11:49:20.769857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769854    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tls-certs\") pod \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\" (UID: \"8ac23fb1-ec62-4e2b-8d43-4288e39eed31\") "
Jul 14 11:49:20.770104 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769951    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8ac23fb1-ec62-4e2b-8d43-4288e39eed31" (UID: "8ac23fb1-ec62-4e2b-8d43-4288e39eed31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:20.770104 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.769956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ac23fb1-ec62-4e2b-8d43-4288e39eed31" (UID: "8ac23fb1-ec62-4e2b-8d43-4288e39eed31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:20.770104 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.770004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-home" (OuterVolumeSpecName: "home") pod "8ac23fb1-ec62-4e2b-8d43-4288e39eed31" (UID: "8ac23fb1-ec62-4e2b-8d43-4288e39eed31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:20.770264 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.770154    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:20.770264 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.770175    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:20.770264 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.770185    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:20.772201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.772072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ac23fb1-ec62-4e2b-8d43-4288e39eed31" (UID: "8ac23fb1-ec62-4e2b-8d43-4288e39eed31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:49:20.772201 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.772151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-dshm" (OuterVolumeSpecName: "dshm") pod "8ac23fb1-ec62-4e2b-8d43-4288e39eed31" (UID: "8ac23fb1-ec62-4e2b-8d43-4288e39eed31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:20.772393 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.772179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-kube-api-access-snqdj" (OuterVolumeSpecName: "kube-api-access-snqdj") pod "8ac23fb1-ec62-4e2b-8d43-4288e39eed31" (UID: "8ac23fb1-ec62-4e2b-8d43-4288e39eed31"). InnerVolumeSpecName "kube-api-access-snqdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:49:20.811375 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.811351    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ac23fb1-ec62-4e2b-8d43-4288e39eed31" containerID="6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1" exitCode=0
Jul 14 11:49:20.811506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.811399    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"
Jul 14 11:49:20.811506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.811412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs" event={"ID":"8ac23fb1-ec62-4e2b-8d43-4288e39eed31","Type":"ContainerDied","Data":"6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1"}
Jul 14 11:49:20.811506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.811446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs" event={"ID":"8ac23fb1-ec62-4e2b-8d43-4288e39eed31","Type":"ContainerDied","Data":"6d288fa6434a20fcc31c647a426a1011bd55927fd6151dd8eb0d024388b29a5d"}
Jul 14 11:49:20.811506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.811461    2563 scope.go:117] "RemoveContainer" containerID="6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1"
Jul 14 11:49:20.820539 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.820521    2563 scope.go:117] "RemoveContainer" containerID="6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1"
Jul 14 11:49:20.820788 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:49:20.820772    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1\": container with ID starting with 6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1 not found: ID does not exist" containerID="6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1"
Jul 14 11:49:20.820850 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.820795    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1"} err="failed to get container status \"6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1\": rpc error: code = NotFound desc = could not find container \"6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1\": container with ID starting with 6f0f218aea481f6195f13c5a12e61002390b0e58c73f6f8406ca0d957b1aeac1 not found: ID does not exist"
Jul 14 11:49:20.833365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.833345    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"]
Jul 14 11:49:20.837590 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.837570    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/auth-test-model-6b8edc-kserve-75f5cfc979-cmnhs"]
Jul 14 11:49:20.870766 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.870720    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:20.870766 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.870737    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:20.870766 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:20.870747    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snqdj\" (UniqueName: \"kubernetes.io/projected/8ac23fb1-ec62-4e2b-8d43-4288e39eed31-kube-api-access-snqdj\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:21.332681 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.332628    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q" podUID="3c02751e-0864-4b10-97cc-546598f90d68" containerName="main" containerID="cri-o://b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366" gracePeriod=45
Jul 14 11:49:21.532130 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.532089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:21.594857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.594804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:21.595929 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.595912    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:49:21.779103 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzdlg\" (UniqueName: \"kubernetes.io/projected/3c02751e-0864-4b10-97cc-546598f90d68-kube-api-access-fzdlg\") pod \"3c02751e-0864-4b10-97cc-546598f90d68\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") "
Jul 14 11:49:21.779293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-home\") pod \"3c02751e-0864-4b10-97cc-546598f90d68\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") "
Jul 14 11:49:21.779293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-tmp-dir\") pod \"3c02751e-0864-4b10-97cc-546598f90d68\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") "
Jul 14 11:49:21.779293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779187    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-dshm\") pod \"3c02751e-0864-4b10-97cc-546598f90d68\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") "
Jul 14 11:49:21.779293 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779249    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-model-cache\") pod \"3c02751e-0864-4b10-97cc-546598f90d68\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") "
Jul 14 11:49:21.779504 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779320    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02751e-0864-4b10-97cc-546598f90d68-tls-certs\") pod \"3c02751e-0864-4b10-97cc-546598f90d68\" (UID: \"3c02751e-0864-4b10-97cc-546598f90d68\") "
Jul 14 11:49:21.779504 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779400    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-home" (OuterVolumeSpecName: "home") pod "3c02751e-0864-4b10-97cc-546598f90d68" (UID: "3c02751e-0864-4b10-97cc-546598f90d68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:21.779504 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779424    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c02751e-0864-4b10-97cc-546598f90d68" (UID: "3c02751e-0864-4b10-97cc-546598f90d68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:21.779679 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c02751e-0864-4b10-97cc-546598f90d68" (UID: "3c02751e-0864-4b10-97cc-546598f90d68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:21.779742 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779702    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:21.779742 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.779721    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:21.781423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.781394    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c02751e-0864-4b10-97cc-546598f90d68-kube-api-access-fzdlg" (OuterVolumeSpecName: "kube-api-access-fzdlg") pod "3c02751e-0864-4b10-97cc-546598f90d68" (UID: "3c02751e-0864-4b10-97cc-546598f90d68"). InnerVolumeSpecName "kube-api-access-fzdlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:49:21.781423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.781406    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-dshm" (OuterVolumeSpecName: "dshm") pod "3c02751e-0864-4b10-97cc-546598f90d68" (UID: "3c02751e-0864-4b10-97cc-546598f90d68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:49:21.781423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.781398    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c02751e-0864-4b10-97cc-546598f90d68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c02751e-0864-4b10-97cc-546598f90d68" (UID: "3c02751e-0864-4b10-97cc-546598f90d68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:49:21.817643 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.817615    2563 generic.go:356] "Generic (PLEG): container finished" podID="3c02751e-0864-4b10-97cc-546598f90d68" containerID="b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366" exitCode=0
Jul 14 11:49:21.817786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.817681    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"
Jul 14 11:49:21.817786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.817707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q" event={"ID":"3c02751e-0864-4b10-97cc-546598f90d68","Type":"ContainerDied","Data":"b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366"}
Jul 14 11:49:21.817786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.817747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q" event={"ID":"3c02751e-0864-4b10-97cc-546598f90d68","Type":"ContainerDied","Data":"b9219ceaae5feee662afbaa0e4e788ce345548877599111c7f38980c52d97485"}
Jul 14 11:49:21.817786 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.817764    2563 scope.go:117] "RemoveContainer" containerID="b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366"
Jul 14 11:49:21.826596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.826574    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ac23fb1-ec62-4e2b-8d43-4288e39eed31" path="/var/lib/kubelet/pods/8ac23fb1-ec62-4e2b-8d43-4288e39eed31/volumes"
Jul 14 11:49:21.826791 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.826737    2563 scope.go:117] "RemoveContainer" containerID="b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366"
Jul 14 11:49:21.827029 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:49:21.827002    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366\": container with ID starting with b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366 not found: ID does not exist" containerID="b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366"
Jul 14 11:49:21.827073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.827042    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366"} err="failed to get container status \"b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366\": rpc error: code = NotFound desc = could not find container \"b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366\": container with ID starting with b2271bcdd83593872f764eb86df8667b5dc1f2257eba8adbb7098f63f986a366 not found: ID does not exist"
Jul 14 11:49:21.839715 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.839693    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"]
Jul 14 11:49:21.843361 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.843340    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/auth-test-model-5c5a7d-kserve-5844846699-4kd5q"]
Jul 14 11:49:21.880248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.880190    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:21.880248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.880211    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02751e-0864-4b10-97cc-546598f90d68-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:21.880248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.880224    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02751e-0864-4b10-97cc-546598f90d68-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:21.880248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:21.880235    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzdlg\" (UniqueName: \"kubernetes.io/projected/3c02751e-0864-4b10-97cc-546598f90d68-kube-api-access-fzdlg\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:49:22.663420 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:22.663386    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:22.663919 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:22.663440    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:22.676107 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:22.676081    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:22.834969 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:22.834937    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:49:23.828616 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:23.828578    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c02751e-0864-4b10-97cc-546598f90d68" path="/var/lib/kubelet/pods/3c02751e-0864-4b10-97cc-546598f90d68/volumes"
Jul 14 11:49:25.446980 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:25.446942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:49:57.644018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.643987    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"]
Jul 14 11:49:57.644472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.644344    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ac23fb1-ec62-4e2b-8d43-4288e39eed31" containerName="main"
Jul 14 11:49:57.644472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.644356    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ac23fb1-ec62-4e2b-8d43-4288e39eed31" containerName="main"
Jul 14 11:49:57.644472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.644367    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c02751e-0864-4b10-97cc-546598f90d68" containerName="main"
Jul 14 11:49:57.644472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.644375    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c02751e-0864-4b10-97cc-546598f90d68" containerName="main"
Jul 14 11:49:57.644472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.644443    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c02751e-0864-4b10-97cc-546598f90d68" containerName="main"
Jul 14 11:49:57.644472 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.644455    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ac23fb1-ec62-4e2b-8d43-4288e39eed31" containerName="main"
Jul 14 11:49:57.647474 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.647456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.652503 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.652479    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"openshift-service-ca.crt\""
Jul 14 11:49:57.652626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.652504    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"kube-root-ca.crt\""
Jul 14 11:49:57.652626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.652481    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"default-dockercfg-6ljq7\""
Jul 14 11:49:57.652626 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.652482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"sub-test-model-5c5a7d-kserve-self-signed-certs\""
Jul 14 11:49:57.660229 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.660208    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"]
Jul 14 11:49:57.694983 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.694953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:49:57.801423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.801332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f3d588-7481-4739-a6d9-46f35142b7a6-tls-certs\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.801423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.801376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-dshm\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.801423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.801410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-home\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.801736 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.801503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-model-cache\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.801736 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.801543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-tmp-dir\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.801736 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.801615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjjz\" (UniqueName: \"kubernetes.io/projected/80f3d588-7481-4739-a6d9-46f35142b7a6-kube-api-access-hdjjz\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.902838 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.902753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjjz\" (UniqueName: \"kubernetes.io/projected/80f3d588-7481-4739-a6d9-46f35142b7a6-kube-api-access-hdjjz\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.902997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.902837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f3d588-7481-4739-a6d9-46f35142b7a6-tls-certs\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.902997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.902878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-dshm\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.902997 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.902922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-home\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.903183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.903076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-model-cache\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.903183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.903131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-tmp-dir\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.903350 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.903328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-home\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.903483 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.903380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-model-cache\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.903483 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.903463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-tmp-dir\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.912031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.905243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-dshm\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.912031 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.905748    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f3d588-7481-4739-a6d9-46f35142b7a6-tls-certs\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.919433 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.919411    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjjz\" (UniqueName: \"kubernetes.io/projected/80f3d588-7481-4739-a6d9-46f35142b7a6-kube-api-access-hdjjz\") pod \"sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") " pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:57.957197 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:57.957172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:49:58.091482 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:58.091454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"]
Jul 14 11:49:58.092844 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:49:58.092819    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80f3d588_7481_4739_a6d9_46f35142b7a6.slice/crio-70476bd64d30e9b5a651bd8702827670d599f5da50d53c53abfb4ecb5fa386cf WatchSource:0}: Error finding container 70476bd64d30e9b5a651bd8702827670d599f5da50d53c53abfb4ecb5fa386cf: Status 404 returned error can't find the container with id 70476bd64d30e9b5a651bd8702827670d599f5da50d53c53abfb4ecb5fa386cf
Jul 14 11:49:58.995728 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:58.995691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb" event={"ID":"80f3d588-7481-4739-a6d9-46f35142b7a6","Type":"ContainerStarted","Data":"e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131"}
Jul 14 11:49:58.996081 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:58.995736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb" event={"ID":"80f3d588-7481-4739-a6d9-46f35142b7a6","Type":"ContainerStarted","Data":"70476bd64d30e9b5a651bd8702827670d599f5da50d53c53abfb4ecb5fa386cf"}
Jul 14 11:49:59.015477 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:49:59.015424    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb" podStartSLOduration=2.015408781 podStartE2EDuration="2.015408781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:49:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:49:59.014848095 +0000 UTC m=+2603.597387102" watchObservedRunningTime="2026-07-14 11:49:59.015408781 +0000 UTC m=+2603.597947764"
Jul 14 11:50:02.452048 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:02.452006    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:50:07.957590 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:07.957546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:50:07.958055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:07.957621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:50:07.970134 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:07.970113    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:50:08.050390 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:08.050360    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:50:21.440809 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:21.440767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:50:21.536647 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:21.536616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:50:26.659904 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:26.659866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:50:32.960696 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:32.960648    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"]
Jul 14 11:50:32.983485 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:32.983458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:50:33.864920 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:33.864889    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"]
Jul 14 11:50:34.478088 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:34.478059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:50:39.851798 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.851763    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"]
Jul 14 11:50:39.855211 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.855191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:39.857876 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.857853    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b8edc\"/\"rate-test-model-6b8edc-kserve-self-signed-certs\""
Jul 14 11:50:39.869101 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.869078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"]
Jul 14 11:50:39.920277 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.920240    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:50:39.970961 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.970932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrt4d\" (UniqueName: \"kubernetes.io/projected/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-kube-api-access-wrt4d\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:39.971154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.970979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tls-certs\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:39.971154 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.971028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-model-cache\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:39.971288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.971162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-dshm\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:39.971288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.971219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tmp-dir\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:39.971288 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:39.971258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-home\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072151 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrt4d\" (UniqueName: \"kubernetes.io/projected/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-kube-api-access-wrt4d\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tls-certs\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-model-cache\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-dshm\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tmp-dir\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-home\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.072838 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-home\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.073018 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.072957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-model-cache\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.073350 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.073253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tmp-dir\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.074903 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.074873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-dshm\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.075379 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.075355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tls-certs\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.081192 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.081164    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrt4d\" (UniqueName: \"kubernetes.io/projected/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-kube-api-access-wrt4d\") pod \"rate-test-model-6b8edc-kserve-6596c8cf47-j22bg\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") " pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.164733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.164618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:40.318834 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:40.318810    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"]
Jul 14 11:50:40.321459 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:50:40.321429    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ce0b972_d8ed_4373_a6c1_3382e83c9cf3.slice/crio-cc53bc6a950408b77ff3d0c2a26bbf0a3e4b9a86f03959f4f9a46b644addbcf7 WatchSource:0}: Error finding container cc53bc6a950408b77ff3d0c2a26bbf0a3e4b9a86f03959f4f9a46b644addbcf7: Status 404 returned error can't find the container with id cc53bc6a950408b77ff3d0c2a26bbf0a3e4b9a86f03959f4f9a46b644addbcf7
Jul 14 11:50:41.196269 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:41.196233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg" event={"ID":"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3","Type":"ContainerStarted","Data":"2f0bdcf3d09297013473e700a2256588b664ae07c8c381f65429f6f11c75e8ca"}
Jul 14 11:50:41.196701 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:41.196275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg" event={"ID":"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3","Type":"ContainerStarted","Data":"cc53bc6a950408b77ff3d0c2a26bbf0a3e4b9a86f03959f4f9a46b644addbcf7"}
Jul 14 11:50:41.215907 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:41.215858    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg" podStartSLOduration=2.215839012 podStartE2EDuration="2.215839012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:50:41.21511648 +0000 UTC m=+2645.797655508" watchObservedRunningTime="2026-07-14 11:50:41.215839012 +0000 UTC m=+2645.798378017"
Jul 14 11:50:46.340696 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:46.340646    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:50:47.981380 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:47.981339    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n" podUID="8be73d31-91eb-49b6-9990-fac5443323aa" containerName="main" containerID="cri-o://6f4c60a8379544f34b8b16a886322b2674f97a14b2a92860014e537e53c50051" gracePeriod=45
Jul 14 11:50:48.231687 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.231606    2563 generic.go:356] "Generic (PLEG): container finished" podID="8be73d31-91eb-49b6-9990-fac5443323aa" containerID="6f4c60a8379544f34b8b16a886322b2674f97a14b2a92860014e537e53c50051" exitCode=0
Jul 14 11:50:48.231789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.231706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n" event={"ID":"8be73d31-91eb-49b6-9990-fac5443323aa","Type":"ContainerDied","Data":"6f4c60a8379544f34b8b16a886322b2674f97a14b2a92860014e537e53c50051"}
Jul 14 11:50:48.231789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.231765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n" event={"ID":"8be73d31-91eb-49b6-9990-fac5443323aa","Type":"ContainerDied","Data":"4714493fc756627989e9525a5f2e34cd30165af8a482d0933d606f4f6bc9fafb"}
Jul 14 11:50:48.231789 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.231784    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4714493fc756627989e9525a5f2e34cd30165af8a482d0933d606f4f6bc9fafb"
Jul 14 11:50:48.238356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.238338    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:50:48.356943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.356906    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-dshm\") pod \"8be73d31-91eb-49b6-9990-fac5443323aa\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") "
Jul 14 11:50:48.356943 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.356950    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-home\") pod \"8be73d31-91eb-49b6-9990-fac5443323aa\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") "
Jul 14 11:50:48.357172 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.356974    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8be73d31-91eb-49b6-9990-fac5443323aa-tls-certs\") pod \"8be73d31-91eb-49b6-9990-fac5443323aa\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") "
Jul 14 11:50:48.357172 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357020    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-model-cache\") pod \"8be73d31-91eb-49b6-9990-fac5443323aa\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") "
Jul 14 11:50:48.357172 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357048    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-tmp-dir\") pod \"8be73d31-91eb-49b6-9990-fac5443323aa\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") "
Jul 14 11:50:48.357172 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6cx5\" (UniqueName: \"kubernetes.io/projected/8be73d31-91eb-49b6-9990-fac5443323aa-kube-api-access-z6cx5\") pod \"8be73d31-91eb-49b6-9990-fac5443323aa\" (UID: \"8be73d31-91eb-49b6-9990-fac5443323aa\") "
Jul 14 11:50:48.357396 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357228    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-home" (OuterVolumeSpecName: "home") pod "8be73d31-91eb-49b6-9990-fac5443323aa" (UID: "8be73d31-91eb-49b6-9990-fac5443323aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:48.357396 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357305    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "8be73d31-91eb-49b6-9990-fac5443323aa" (UID: "8be73d31-91eb-49b6-9990-fac5443323aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:48.357396 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357382    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8be73d31-91eb-49b6-9990-fac5443323aa" (UID: "8be73d31-91eb-49b6-9990-fac5443323aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:48.357513 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357426    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:48.357513 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.357445    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:48.359041 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.359009    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-dshm" (OuterVolumeSpecName: "dshm") pod "8be73d31-91eb-49b6-9990-fac5443323aa" (UID: "8be73d31-91eb-49b6-9990-fac5443323aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:48.359171 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.359094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8be73d31-91eb-49b6-9990-fac5443323aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8be73d31-91eb-49b6-9990-fac5443323aa" (UID: "8be73d31-91eb-49b6-9990-fac5443323aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:50:48.359171 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.359098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be73d31-91eb-49b6-9990-fac5443323aa-kube-api-access-z6cx5" (OuterVolumeSpecName: "kube-api-access-z6cx5") pod "8be73d31-91eb-49b6-9990-fac5443323aa" (UID: "8be73d31-91eb-49b6-9990-fac5443323aa"). InnerVolumeSpecName "kube-api-access-z6cx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:50:48.458537 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.458504    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:48.458537 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.458533    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8be73d31-91eb-49b6-9990-fac5443323aa-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:48.458537 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.458543    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8be73d31-91eb-49b6-9990-fac5443323aa-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:48.458792 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.458553    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6cx5\" (UniqueName: \"kubernetes.io/projected/8be73d31-91eb-49b6-9990-fac5443323aa-kube-api-access-z6cx5\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:48.887397 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:48.887360    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb" podUID="80f3d588-7481-4739-a6d9-46f35142b7a6" containerName="main" containerID="cri-o://e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131" gracePeriod=45
Jul 14 11:50:49.135247 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.135221    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:50:49.237073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.236994    2563 generic.go:356] "Generic (PLEG): container finished" podID="80f3d588-7481-4739-a6d9-46f35142b7a6" containerID="e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131" exitCode=0
Jul 14 11:50:49.237073 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.237037    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"
Jul 14 11:50:49.237248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.237076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb" event={"ID":"80f3d588-7481-4739-a6d9-46f35142b7a6","Type":"ContainerDied","Data":"e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131"}
Jul 14 11:50:49.237248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.237114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb" event={"ID":"80f3d588-7481-4739-a6d9-46f35142b7a6","Type":"ContainerDied","Data":"70476bd64d30e9b5a651bd8702827670d599f5da50d53c53abfb4ecb5fa386cf"}
Jul 14 11:50:49.237248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.237131    2563 scope.go:117] "RemoveContainer" containerID="e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131"
Jul 14 11:50:49.237248 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.237175    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"
Jul 14 11:50:49.246025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.246010    2563 scope.go:117] "RemoveContainer" containerID="e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131"
Jul 14 11:50:49.246264 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:50:49.246246    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131\": container with ID starting with e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131 not found: ID does not exist" containerID="e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131"
Jul 14 11:50:49.246334 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.246270    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131"} err="failed to get container status \"e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131\": rpc error: code = NotFound desc = could not find container \"e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131\": container with ID starting with e0d5759dd3609e66ec0a6e8034fba0aa5c1c1cd8736bda6d60d45d07c55f8131 not found: ID does not exist"
Jul 14 11:50:49.264326 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.264285    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"]
Jul 14 11:50:49.264849 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.264830    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-home\") pod \"80f3d588-7481-4739-a6d9-46f35142b7a6\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") "
Jul 14 11:50:49.264913 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.264860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-dshm\") pod \"80f3d588-7481-4739-a6d9-46f35142b7a6\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") "
Jul 14 11:50:49.264913 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.264887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-model-cache\") pod \"80f3d588-7481-4739-a6d9-46f35142b7a6\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") "
Jul 14 11:50:49.264981 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.264921    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-tmp-dir\") pod \"80f3d588-7481-4739-a6d9-46f35142b7a6\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") "
Jul 14 11:50:49.265136 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265118    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdjjz\" (UniqueName: \"kubernetes.io/projected/80f3d588-7481-4739-a6d9-46f35142b7a6-kube-api-access-hdjjz\") pod \"80f3d588-7481-4739-a6d9-46f35142b7a6\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") "
Jul 14 11:50:49.265216 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265147    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/sub-test-model-6b8edc-kserve-767f6c8df9-lb66n"]
Jul 14 11:50:49.265216 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f3d588-7481-4739-a6d9-46f35142b7a6-tls-certs\") pod \"80f3d588-7481-4739-a6d9-46f35142b7a6\" (UID: \"80f3d588-7481-4739-a6d9-46f35142b7a6\") "
Jul 14 11:50:49.265315 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265239    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-home" (OuterVolumeSpecName: "home") pod "80f3d588-7481-4739-a6d9-46f35142b7a6" (UID: "80f3d588-7481-4739-a6d9-46f35142b7a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:49.265368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265309    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "80f3d588-7481-4739-a6d9-46f35142b7a6" (UID: "80f3d588-7481-4739-a6d9-46f35142b7a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:49.265368 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80f3d588-7481-4739-a6d9-46f35142b7a6" (UID: "80f3d588-7481-4739-a6d9-46f35142b7a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:49.265503 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265482    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:49.265503 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265504    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:49.265640 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.265518    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:49.267123 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.267093    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-dshm" (OuterVolumeSpecName: "dshm") pod "80f3d588-7481-4739-a6d9-46f35142b7a6" (UID: "80f3d588-7481-4739-a6d9-46f35142b7a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:50:49.267218 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.267164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f3d588-7481-4739-a6d9-46f35142b7a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80f3d588-7481-4739-a6d9-46f35142b7a6" (UID: "80f3d588-7481-4739-a6d9-46f35142b7a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:50:49.267405 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.267385    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f3d588-7481-4739-a6d9-46f35142b7a6-kube-api-access-hdjjz" (OuterVolumeSpecName: "kube-api-access-hdjjz") pod "80f3d588-7481-4739-a6d9-46f35142b7a6" (UID: "80f3d588-7481-4739-a6d9-46f35142b7a6"). InnerVolumeSpecName "kube-api-access-hdjjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:50:49.366503 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.366465    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdjjz\" (UniqueName: \"kubernetes.io/projected/80f3d588-7481-4739-a6d9-46f35142b7a6-kube-api-access-hdjjz\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:49.366503 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.366498    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f3d588-7481-4739-a6d9-46f35142b7a6-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:49.366503 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.366509    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f3d588-7481-4739-a6d9-46f35142b7a6-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:50:49.561880 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.561849    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"]
Jul 14 11:50:49.567577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.567552    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/sub-test-model-5c5a7d-kserve-759f46bc69-h9ljb"]
Jul 14 11:50:49.825925 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.825842    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80f3d588-7481-4739-a6d9-46f35142b7a6" path="/var/lib/kubelet/pods/80f3d588-7481-4739-a6d9-46f35142b7a6/volumes"
Jul 14 11:50:49.826235 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:49.826222    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8be73d31-91eb-49b6-9990-fac5443323aa" path="/var/lib/kubelet/pods/8be73d31-91eb-49b6-9990-fac5443323aa/volumes"
Jul 14 11:50:50.165717 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:50.165615    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:50.165717 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:50.165687    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:50.181301 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:50.181272    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:50:50.254178 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:50:50.254151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:51:10.855281 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855248    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"]
Jul 14 11:51:10.855763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80f3d588-7481-4739-a6d9-46f35142b7a6" containerName="main"
Jul 14 11:51:10.855763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855604    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="80f3d588-7481-4739-a6d9-46f35142b7a6" containerName="main"
Jul 14 11:51:10.855763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855622    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be73d31-91eb-49b6-9990-fac5443323aa" containerName="main"
Jul 14 11:51:10.855763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855629    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be73d31-91eb-49b6-9990-fac5443323aa" containerName="main"
Jul 14 11:51:10.855763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855711    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be73d31-91eb-49b6-9990-fac5443323aa" containerName="main"
Jul 14 11:51:10.855763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.855719    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="80f3d588-7481-4739-a6d9-46f35142b7a6" containerName="main"
Jul 14 11:51:10.860210 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.860190    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:10.862800 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.862782    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"openshift-service-ca.crt\""
Jul 14 11:51:10.863639 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.863624    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"default-dockercfg-6ljq7\""
Jul 14 11:51:10.863733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.863699    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"rate-test-model-5c5a7d-kserve-self-signed-certs\""
Jul 14 11:51:10.863733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.863719    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c5a7d\"/\"kube-root-ca.crt\""
Jul 14 11:51:10.873504 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.873485    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"]
Jul 14 11:51:10.901891 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.901864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:51:10.948197 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.948166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jw4l\" (UniqueName: \"kubernetes.io/projected/a4896620-e8ca-4461-8e5a-e4380f789dbe-kube-api-access-5jw4l\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:10.948356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.948200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-tmp-dir\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:10.948356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.948235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-dshm\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:10.948356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.948253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4896620-e8ca-4461-8e5a-e4380f789dbe-tls-certs\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:10.948356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.948309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-home\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:10.948356 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:10.948332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-model-cache\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.049572 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.049536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jw4l\" (UniqueName: \"kubernetes.io/projected/a4896620-e8ca-4461-8e5a-e4380f789dbe-kube-api-access-5jw4l\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.049782 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.049583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-tmp-dir\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.049782 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.049719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-dshm\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.049782 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.049764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4896620-e8ca-4461-8e5a-e4380f789dbe-tls-certs\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.049954 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.049869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-home\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.049954 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.049906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-model-cache\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.050337 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.050278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-home\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.050337 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.050285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-tmp-dir\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.050485 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.050383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-model-cache\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.052202 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.052173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-dshm\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.052607 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.052586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4896620-e8ca-4461-8e5a-e4380f789dbe-tls-certs\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.059843 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.059819    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jw4l\" (UniqueName: \"kubernetes.io/projected/a4896620-e8ca-4461-8e5a-e4380f789dbe-kube-api-access-5jw4l\") pod \"rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") " pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.169577 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.169474    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:11.305821 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.305793    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"]
Jul 14 11:51:11.307481 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:51:11.307453    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4896620_e8ca_4461_8e5a_e4380f789dbe.slice/crio-34e5b7e0d648c41b0654ab18c194fe28b94bde618d5699a83b32546c6d2f59a2 WatchSource:0}: Error finding container 34e5b7e0d648c41b0654ab18c194fe28b94bde618d5699a83b32546c6d2f59a2: Status 404 returned error can't find the container with id 34e5b7e0d648c41b0654ab18c194fe28b94bde618d5699a83b32546c6d2f59a2
Jul 14 11:51:11.347505 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:11.347471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf" event={"ID":"a4896620-e8ca-4461-8e5a-e4380f789dbe","Type":"ContainerStarted","Data":"34e5b7e0d648c41b0654ab18c194fe28b94bde618d5699a83b32546c6d2f59a2"}
Jul 14 11:51:12.353861 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:12.353824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf" event={"ID":"a4896620-e8ca-4461-8e5a-e4380f789dbe","Type":"ContainerStarted","Data":"1d0e88b2ac692a0c1f5de982445c9ab295b1d6aa6826cafad8295f2812371074"}
Jul 14 11:51:12.374077 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:12.374030    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf" podStartSLOduration=2.374010376 podStartE2EDuration="2.374010376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:51:12.372958419 +0000 UTC m=+2676.955497420" watchObservedRunningTime="2026-07-14 11:51:12.374010376 +0000 UTC m=+2676.956549381"
Jul 14 11:51:15.281641 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:15.281604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:51:21.170370 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:21.170335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:21.170795 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:21.170414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:21.187525 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:21.187499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:21.413668 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:21.413626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:51:35.973877 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:35.973793    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:51:35.978820 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:35.978796    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5675s_47ba1577-742a-426c-ad17-aecc7e57e72e/ovn-acl-logging/0.log"
Jul 14 11:51:38.399822 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:38.399786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:51:38.503686 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:38.503638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:51:39.334787 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:39.334755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:51:46.551460 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:46.551411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:51:46.652760 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:46.652728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:51:54.740033 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:51:54.739985    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:52:00.513522 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:00.513488    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"]
Jul 14 11:52:00.534610 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:00.534568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:01.447025 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:01.446981    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"]
Jul 14 11:52:01.471961 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:01.471926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:12.343461 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:12.343428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:52:15.533569 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.533511    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg" podUID="8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" containerName="main" containerID="cri-o://2f0bdcf3d09297013473e700a2256588b664ae07c8c381f65429f6f11c75e8ca" gracePeriod=45
Jul 14 11:52:15.669531 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.669502    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" containerID="2f0bdcf3d09297013473e700a2256588b664ae07c8c381f65429f6f11c75e8ca" exitCode=0
Jul 14 11:52:15.669709 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.669583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg" event={"ID":"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3","Type":"ContainerDied","Data":"2f0bdcf3d09297013473e700a2256588b664ae07c8c381f65429f6f11c75e8ca"}
Jul 14 11:52:15.788246 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.788186    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:52:15.833055 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tmp-dir\") pod \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") "
Jul 14 11:52:15.833183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833070    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-model-cache\") pod \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") "
Jul 14 11:52:15.833183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833092    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-dshm\") pod \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") "
Jul 14 11:52:15.833183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833138    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tls-certs\") pod \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") "
Jul 14 11:52:15.833183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-home\") pod \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") "
Jul 14 11:52:15.833715 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833564    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrt4d\" (UniqueName: \"kubernetes.io/projected/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-kube-api-access-wrt4d\") pod \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\" (UID: \"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3\") "
Jul 14 11:52:15.833715 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833604    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" (UID: "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:15.833928 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.833738    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" (UID: "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:15.834027 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.834002    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-home" (OuterVolumeSpecName: "home") pod "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" (UID: "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:15.834027 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.834017    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:15.834214 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.834047    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:15.841175 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.835958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-dshm" (OuterVolumeSpecName: "dshm") pod "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" (UID: "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:15.841741 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.841606    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" (UID: "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:52:15.841874 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.841743    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-kube-api-access-wrt4d" (OuterVolumeSpecName: "kube-api-access-wrt4d") pod "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" (UID: "8ce0b972-d8ed-4373-a6c1-3382e83c9cf3"). InnerVolumeSpecName "kube-api-access-wrt4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:52:15.934910 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.934879    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrt4d\" (UniqueName: \"kubernetes.io/projected/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-kube-api-access-wrt4d\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:15.934910 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.934908    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:15.935134 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.934922    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:15.935134 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:15.934931    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:16.466807 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.466760    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf" podUID="a4896620-e8ca-4461-8e5a-e4380f789dbe" containerName="main" containerID="cri-o://1d0e88b2ac692a0c1f5de982445c9ab295b1d6aa6826cafad8295f2812371074" gracePeriod=45
Jul 14 11:52:16.678756 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.678637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg" event={"ID":"8ce0b972-d8ed-4373-a6c1-3382e83c9cf3","Type":"ContainerDied","Data":"cc53bc6a950408b77ff3d0c2a26bbf0a3e4b9a86f03959f4f9a46b644addbcf7"}
Jul 14 11:52:16.678756 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.678674    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"
Jul 14 11:52:16.678756 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.678734    2563 scope.go:117] "RemoveContainer" containerID="2f0bdcf3d09297013473e700a2256588b664ae07c8c381f65429f6f11c75e8ca"
Jul 14 11:52:16.680638 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.680610    2563 generic.go:356] "Generic (PLEG): container finished" podID="a4896620-e8ca-4461-8e5a-e4380f789dbe" containerID="1d0e88b2ac692a0c1f5de982445c9ab295b1d6aa6826cafad8295f2812371074" exitCode=0
Jul 14 11:52:16.680779 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.680697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf" event={"ID":"a4896620-e8ca-4461-8e5a-e4380f789dbe","Type":"ContainerDied","Data":"1d0e88b2ac692a0c1f5de982445c9ab295b1d6aa6826cafad8295f2812371074"}
Jul 14 11:52:16.702056 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.702030    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"]
Jul 14 11:52:16.706778 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.706748    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6b8edc/rate-test-model-6b8edc-kserve-6596c8cf47-j22bg"]
Jul 14 11:52:16.729278 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.729225    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:52:16.843117 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-home\") pod \"a4896620-e8ca-4461-8e5a-e4380f789dbe\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") "
Jul 14 11:52:16.843313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jw4l\" (UniqueName: \"kubernetes.io/projected/a4896620-e8ca-4461-8e5a-e4380f789dbe-kube-api-access-5jw4l\") pod \"a4896620-e8ca-4461-8e5a-e4380f789dbe\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") "
Jul 14 11:52:16.843313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843166    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4896620-e8ca-4461-8e5a-e4380f789dbe-tls-certs\") pod \"a4896620-e8ca-4461-8e5a-e4380f789dbe\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") "
Jul 14 11:52:16.843313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843206    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-model-cache\") pod \"a4896620-e8ca-4461-8e5a-e4380f789dbe\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") "
Jul 14 11:52:16.843313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843247    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-dshm\") pod \"a4896620-e8ca-4461-8e5a-e4380f789dbe\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") "
Jul 14 11:52:16.843313 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-tmp-dir\") pod \"a4896620-e8ca-4461-8e5a-e4380f789dbe\" (UID: \"a4896620-e8ca-4461-8e5a-e4380f789dbe\") "
Jul 14 11:52:16.843596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843369    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-home" (OuterVolumeSpecName: "home") pod "a4896620-e8ca-4461-8e5a-e4380f789dbe" (UID: "a4896620-e8ca-4461-8e5a-e4380f789dbe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:16.843596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4896620-e8ca-4461-8e5a-e4380f789dbe" (UID: "a4896620-e8ca-4461-8e5a-e4380f789dbe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:16.843596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843543    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-home\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:16.843596 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.843589    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4896620-e8ca-4461-8e5a-e4380f789dbe" (UID: "a4896620-e8ca-4461-8e5a-e4380f789dbe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:16.845379 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.845355    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4896620-e8ca-4461-8e5a-e4380f789dbe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4896620-e8ca-4461-8e5a-e4380f789dbe" (UID: "a4896620-e8ca-4461-8e5a-e4380f789dbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:52:16.845506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.845374    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-dshm" (OuterVolumeSpecName: "dshm") pod "a4896620-e8ca-4461-8e5a-e4380f789dbe" (UID: "a4896620-e8ca-4461-8e5a-e4380f789dbe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:52:16.845506 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.845420    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4896620-e8ca-4461-8e5a-e4380f789dbe-kube-api-access-5jw4l" (OuterVolumeSpecName: "kube-api-access-5jw4l") pod "a4896620-e8ca-4461-8e5a-e4380f789dbe" (UID: "a4896620-e8ca-4461-8e5a-e4380f789dbe"). InnerVolumeSpecName "kube-api-access-5jw4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:52:16.944752 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.944712    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-dshm\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:16.944752 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.944748    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-tmp-dir\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:16.944752 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.944760    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jw4l\" (UniqueName: \"kubernetes.io/projected/a4896620-e8ca-4461-8e5a-e4380f789dbe-kube-api-access-5jw4l\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:16.944996 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.944772    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4896620-e8ca-4461-8e5a-e4380f789dbe-tls-certs\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:16.944996 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:16.944784    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4896620-e8ca-4461-8e5a-e4380f789dbe-model-cache\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:17.689608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.689579    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"
Jul 14 11:52:17.689608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.689594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf" event={"ID":"a4896620-e8ca-4461-8e5a-e4380f789dbe","Type":"ContainerDied","Data":"34e5b7e0d648c41b0654ab18c194fe28b94bde618d5699a83b32546c6d2f59a2"}
Jul 14 11:52:17.690234 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.689671    2563 scope.go:117] "RemoveContainer" containerID="1d0e88b2ac692a0c1f5de982445c9ab295b1d6aa6826cafad8295f2812371074"
Jul 14 11:52:17.714041 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.714010    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"]
Jul 14 11:52:17.716090 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.716066    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5c5a7d/rate-test-model-5c5a7d-kserve-74d7d5db4c-sr6tf"]
Jul 14 11:52:17.826343 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.826304    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" path="/var/lib/kubelet/pods/8ce0b972-d8ed-4373-a6c1-3382e83c9cf3/volumes"
Jul 14 11:52:17.826812 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:17.826791    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4896620-e8ca-4461-8e5a-e4380f789dbe" path="/var/lib/kubelet/pods/a4896620-e8ca-4461-8e5a-e4380f789dbe/volumes"
Jul 14 11:52:18.028467 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028427    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7757-140190212301856"]
Jul 14 11:52:18.028796 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028783    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4896620-e8ca-4461-8e5a-e4380f789dbe" containerName="main"
Jul 14 11:52:18.028858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028797    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4896620-e8ca-4461-8e5a-e4380f789dbe" containerName="main"
Jul 14 11:52:18.028858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028818    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" containerName="main"
Jul 14 11:52:18.028858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028823    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" containerName="main"
Jul 14 11:52:18.028965 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028882    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4896620-e8ca-4461-8e5a-e4380f789dbe" containerName="main"
Jul 14 11:52:18.028965 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.028892    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce0b972-d8ed-4373-a6c1-3382e83c9cf3" containerName="main"
Jul 14 11:52:18.032792 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.032776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:18.035415 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.035393    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8dvv5\""
Jul 14 11:52:18.038475 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.038446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7757-140190212301856"]
Jul 14 11:52:18.153751 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.153717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzq6h\" (UniqueName: \"kubernetes.io/projected/c4ce006f-1068-4327-9a22-1daed828e2a2-kube-api-access-zzq6h\") pod \"test-curl-7757-140190212301856\" (UID: \"c4ce006f-1068-4327-9a22-1daed828e2a2\") " pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:18.254782 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.254741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzq6h\" (UniqueName: \"kubernetes.io/projected/c4ce006f-1068-4327-9a22-1daed828e2a2-kube-api-access-zzq6h\") pod \"test-curl-7757-140190212301856\" (UID: \"c4ce006f-1068-4327-9a22-1daed828e2a2\") " pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:18.263906 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.263874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzq6h\" (UniqueName: \"kubernetes.io/projected/c4ce006f-1068-4327-9a22-1daed828e2a2-kube-api-access-zzq6h\") pod \"test-curl-7757-140190212301856\" (UID: \"c4ce006f-1068-4327-9a22-1daed828e2a2\") " pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:18.342529 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.342436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:18.483127 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.483094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7757-140190212301856"]
Jul 14 11:52:18.486906 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:52:18.486881    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4ce006f_1068_4327_9a22_1daed828e2a2.slice/crio-3c6841f78e130edc0353ef218f864833640cb13d52f15d9b300f63509c737e41 WatchSource:0}: Error finding container 3c6841f78e130edc0353ef218f864833640cb13d52f15d9b300f63509c737e41: Status 404 returned error can't find the container with id 3c6841f78e130edc0353ef218f864833640cb13d52f15d9b300f63509c737e41
Jul 14 11:52:18.488230 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.488208    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:52:18.698185 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:18.698099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7757-140190212301856" event={"ID":"c4ce006f-1068-4327-9a22-1daed828e2a2","Type":"ContainerStarted","Data":"3c6841f78e130edc0353ef218f864833640cb13d52f15d9b300f63509c737e41"}
Jul 14 11:52:19.704341 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:19.704304    2563 generic.go:356] "Generic (PLEG): container finished" podID="c4ce006f-1068-4327-9a22-1daed828e2a2" containerID="665211ba3ad14402188579da00b86817ad4f663dc893eecc173cc051fd31f411" exitCode=0
Jul 14 11:52:19.704763 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:19.704399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7757-140190212301856" event={"ID":"c4ce006f-1068-4327-9a22-1daed828e2a2","Type":"ContainerDied","Data":"665211ba3ad14402188579da00b86817ad4f663dc893eecc173cc051fd31f411"}
Jul 14 11:52:20.843978 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:20.843948    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:20.865084 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:20.865060    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7757-140190212301856_c4ce006f-1068-4327-9a22-1daed828e2a2/test-curl-7757-140190212301856/0.log"
Jul 14 11:52:20.888901 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:20.888871    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7757-140190212301856"]
Jul 14 11:52:20.892733 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:20.892710    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7757-140190212301856"]
Jul 14 11:52:20.978894 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:20.978801    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzq6h\" (UniqueName: \"kubernetes.io/projected/c4ce006f-1068-4327-9a22-1daed828e2a2-kube-api-access-zzq6h\") pod \"c4ce006f-1068-4327-9a22-1daed828e2a2\" (UID: \"c4ce006f-1068-4327-9a22-1daed828e2a2\") "
Jul 14 11:52:20.981037 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:20.981004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ce006f-1068-4327-9a22-1daed828e2a2-kube-api-access-zzq6h" (OuterVolumeSpecName: "kube-api-access-zzq6h") pod "c4ce006f-1068-4327-9a22-1daed828e2a2" (UID: "c4ce006f-1068-4327-9a22-1daed828e2a2"). InnerVolumeSpecName "kube-api-access-zzq6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:52:21.079781 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:21.079743    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzq6h\" (UniqueName: \"kubernetes.io/projected/c4ce006f-1068-4327-9a22-1daed828e2a2-kube-api-access-zzq6h\") on node \"ip-10-0-137-65.ec2.internal\" DevicePath \"\""
Jul 14 11:52:21.716338 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:21.716310    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c6841f78e130edc0353ef218f864833640cb13d52f15d9b300f63509c737e41"
Jul 14 11:52:21.716338 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:21.716308    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:21.716935 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:21.716910    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:21.727155 ip-10-0-137-65 kubenswrapper[2563]: E0714 11:52:21.727129    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7757-140190212301856\" is forbidden: User \"system:node:ip-10-0-137-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-65.ec2.internal' and this object" podUID="c4ce006f-1068-4327-9a22-1daed828e2a2" pod="opendatahub/test-curl-7757-140190212301856"
Jul 14 11:52:21.811743 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:21.811682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:21.825389 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:21.825350    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ce006f-1068-4327-9a22-1daed828e2a2" path="/var/lib/kubelet/pods/c4ce006f-1068-4327-9a22-1daed828e2a2/volumes"
Jul 14 11:52:31.638957 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:31.638921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:34.843120 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:34.843081    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:52:36.248050 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:36.248015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:42.658864 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:42.658830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:42.758173 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:42.758139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:47.078240 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:47.078197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:47.344033 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:47.343948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:48.217009 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:48.216978    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:48.326806 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:48.326772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:52:58.056004 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:52:58.055920    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:01.155857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:01.155822    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fx27v"]
Jul 14 11:53:02.583441 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:02.583408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:08.907527 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:08.907501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:08.999600 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:08.999564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:13.331568 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:13.331534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:13.584602 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:13.584514    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:14.005175 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:14.005100    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-854559bc69-zn6gc_babebe6f-47b8-42a6-a170-421fde3c0e0b/authorino/0.log"
Jul 14 11:53:19.235196 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:19.235162    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kgxqw_fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e/manager/0.log"
Jul 14 11:53:19.355700 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:19.355669    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lrbwz_f85bd830-b0f6-4acd-af45-9434c65e1b57/manager/0.log"
Jul 14 11:53:19.608497 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:19.608475    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ck7lx_2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0/server/0.log"
Jul 14 11:53:19.730461 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:19.730434    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9qchz_a9eefb21-528a-4f63-b078-8f1032c8c5b7/manager/0.log"
Jul 14 11:53:19.985630 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:19.985555    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-rhh5r_44b2011e-a297-4b0a-9c8f-4749b188b83f/copy-manifests/0.log"
Jul 14 11:53:19.994000 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:19.993969    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-rhh5r_44b2011e-a297-4b0a-9c8f-4749b188b83f/manager/0.log"
Jul 14 11:53:23.763414 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:23.763384    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-854559bc69-zn6gc_babebe6f-47b8-42a6-a170-421fde3c0e0b/authorino/0.log"
Jul 14 11:53:23.893000 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:23.892969    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rm29r_3811920d-ef8b-4c59-8214-ab9889832643/manager/0.log"
Jul 14 11:53:24.015817 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:24.015737    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j97kn_4a8ee53b-dcc1-41ba-9a61-e7078fa1e441/manager/0.log"
Jul 14 11:53:24.263783 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:24.263750    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b4zdd_2ba97b27-5bbe-4b3e-b34a-57be9eca86dc/registry-server/0.log"
Jul 14 11:53:24.399148 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:24.399116    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zlmdb_de9bebf1-1874-40a0-9e1c-c6065aaababe/manager/0.log"
Jul 14 11:53:24.538145 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:24.538115    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-fx27v_7307c388-ffa8-4a5b-86ed-1bcd809226d7/limitador/0.log"
Jul 14 11:53:24.682307 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:24.682228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9vv54_f8569c6f-8fd6-4487-b579-bf30da3823ef/manager/0.log"
Jul 14 11:53:25.053423 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:25.053394    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jm7x5_2f8ed243-8f8c-4bd8-9525-5e403805fa17/istio-proxy/0.log"
Jul 14 11:53:25.424181 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:25.424097    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f79597649-t4bls_177c1b47-85e9-4844-9c70-a845b2d429d3/kube-auth-proxy/0.log"
Jul 14 11:53:25.547896 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:25.547869    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/1.log"
Jul 14 11:53:26.879183 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:26.879152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-98x4r_7d56d27a-bff6-4450-82a5-799ad65fd08d/main/0.log"
Jul 14 11:53:30.936886 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.936858    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j46tr/must-gather-dfhxg"]
Jul 14 11:53:30.937341 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.937208    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ce006f-1068-4327-9a22-1daed828e2a2" containerName="test-curl-7757-140190212301856"
Jul 14 11:53:30.937341 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.937220    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ce006f-1068-4327-9a22-1daed828e2a2" containerName="test-curl-7757-140190212301856"
Jul 14 11:53:30.937341 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.937283    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ce006f-1068-4327-9a22-1daed828e2a2" containerName="test-curl-7757-140190212301856"
Jul 14 11:53:30.940722 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.940692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:30.943867 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.943847    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j46tr\"/\"default-dockercfg-ntl7c\""
Jul 14 11:53:30.944290 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.944273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j46tr\"/\"openshift-service-ca.crt\""
Jul 14 11:53:30.944414 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.944396    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j46tr\"/\"kube-root-ca.crt\""
Jul 14 11:53:30.951619 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.951593    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j46tr/must-gather-dfhxg"]
Jul 14 11:53:30.960933 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:30.960910    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:31.042465 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.042414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:31.072358 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.072327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5h85\" (UniqueName: \"kubernetes.io/projected/cec55e29-e952-41ce-8a09-9a31f0a1694b-kube-api-access-v5h85\") pod \"must-gather-dfhxg\" (UID: \"cec55e29-e952-41ce-8a09-9a31f0a1694b\") " pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.072543 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.072374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cec55e29-e952-41ce-8a09-9a31f0a1694b-must-gather-output\") pod \"must-gather-dfhxg\" (UID: \"cec55e29-e952-41ce-8a09-9a31f0a1694b\") " pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.173859 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.173825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5h85\" (UniqueName: \"kubernetes.io/projected/cec55e29-e952-41ce-8a09-9a31f0a1694b-kube-api-access-v5h85\") pod \"must-gather-dfhxg\" (UID: \"cec55e29-e952-41ce-8a09-9a31f0a1694b\") " pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.174019 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.173891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cec55e29-e952-41ce-8a09-9a31f0a1694b-must-gather-output\") pod \"must-gather-dfhxg\" (UID: \"cec55e29-e952-41ce-8a09-9a31f0a1694b\") " pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.174235 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.174214    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cec55e29-e952-41ce-8a09-9a31f0a1694b-must-gather-output\") pod \"must-gather-dfhxg\" (UID: \"cec55e29-e952-41ce-8a09-9a31f0a1694b\") " pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.182959 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.182938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5h85\" (UniqueName: \"kubernetes.io/projected/cec55e29-e952-41ce-8a09-9a31f0a1694b-kube-api-access-v5h85\") pod \"must-gather-dfhxg\" (UID: \"cec55e29-e952-41ce-8a09-9a31f0a1694b\") " pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.249965 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.249872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j46tr/must-gather-dfhxg"
Jul 14 11:53:31.387798 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:31.387766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j46tr/must-gather-dfhxg"]
Jul 14 11:53:31.390513 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:53:31.390485    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcec55e29_e952_41ce_8a09_9a31f0a1694b.slice/crio-2ec5461d54af7fb950517ff769b3580f78367763834a767fb47950f26149cb33 WatchSource:0}: Error finding container 2ec5461d54af7fb950517ff769b3580f78367763834a767fb47950f26149cb33: Status 404 returned error can't find the container with id 2ec5461d54af7fb950517ff769b3580f78367763834a767fb47950f26149cb33
Jul 14 11:53:32.053593 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:32.053555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/must-gather-dfhxg" event={"ID":"cec55e29-e952-41ce-8a09-9a31f0a1694b","Type":"ContainerStarted","Data":"2ec5461d54af7fb950517ff769b3580f78367763834a767fb47950f26149cb33"}
Jul 14 11:53:33.064694 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:33.064622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/must-gather-dfhxg" event={"ID":"cec55e29-e952-41ce-8a09-9a31f0a1694b","Type":"ContainerStarted","Data":"bb527bffb5278c10420c327360036e6271a633541463e85df47429129b46f3b2"}
Jul 14 11:53:33.064694 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:33.064697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/must-gather-dfhxg" event={"ID":"cec55e29-e952-41ce-8a09-9a31f0a1694b","Type":"ContainerStarted","Data":"a6306e7f099675d2598f077a5fe7cb7ee51a4c48b1d00c94b061de94b610e92b"}
Jul 14 11:53:33.085585 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:33.085525    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j46tr/must-gather-dfhxg" podStartSLOduration=2.262407076 podStartE2EDuration="3.085504982s" totalImagesPullingTime="823.097906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:53:33.084181984 +0000 UTC m=+2817.666720991" watchObservedRunningTime="2026-07-14 11:53:33.085504982 +0000 UTC m=+2817.668043988"
Jul 14 11:53:33.907858 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:33.907793    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tb8l9_c3e6c755-ee41-48b9-81d4-7caf68a60441/global-pull-secret-syncer/0.log"
Jul 14 11:53:33.964037 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:33.964003    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log"
Jul 14 11:53:34.114949 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:34.114918    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-65.ec2.internal_ace48c1090b8dbb21683e27f4527c23e/haproxy/0.log"
Jul 14 11:53:39.557273 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:39.557227    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-854559bc69-zn6gc_babebe6f-47b8-42a6-a170-421fde3c0e0b/authorino/0.log"
Jul 14 11:53:39.630612 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:39.630581    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rm29r_3811920d-ef8b-4c59-8214-ab9889832643/manager/0.log"
Jul 14 11:53:39.690806 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:39.690726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j97kn_4a8ee53b-dcc1-41ba-9a61-e7078fa1e441/manager/0.log"
Jul 14 11:53:39.783292 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:39.783217    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b4zdd_2ba97b27-5bbe-4b3e-b34a-57be9eca86dc/registry-server/0.log"
Jul 14 11:53:39.965145 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:39.965056    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zlmdb_de9bebf1-1874-40a0-9e1c-c6065aaababe/manager/0.log"
Jul 14 11:53:39.994442 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:39.994405    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-fx27v_7307c388-ffa8-4a5b-86ed-1bcd809226d7/limitador/0.log"
Jul 14 11:53:40.080677 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:40.080036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9vv54_f8569c6f-8fd6-4487-b579-bf30da3823ef/manager/0.log"
Jul 14 11:53:42.288361 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:42.288333    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vpx7j_7e415343-0445-463e-9139-b8b47765a898/node-exporter/0.log"
Jul 14 11:53:42.315697 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:42.315628    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vpx7j_7e415343-0445-463e-9139-b8b47765a898/kube-rbac-proxy/0.log"
Jul 14 11:53:42.343857 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:42.343834    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vpx7j_7e415343-0445-463e-9139-b8b47765a898/init-textfile/0.log"
Jul 14 11:53:43.543922 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.543877    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"]
Jul 14 11:53:43.550953 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.550922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.582980 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.582937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"]
Jul 14 11:53:43.597264 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.597225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-lib-modules\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.597460 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.597283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqqtq\" (UniqueName: \"kubernetes.io/projected/b21010b5-5405-4fa0-b2b1-26f8b760c155-kube-api-access-tqqtq\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.597460 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.597351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-proc\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.597608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.597478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-sys\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.597608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.597542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-podres\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698412 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-sys\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-podres\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-lib-modules\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-sys\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698583 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqtq\" (UniqueName: \"kubernetes.io/projected/b21010b5-5405-4fa0-b2b1-26f8b760c155-kube-api-access-tqqtq\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-proc\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-podres\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-lib-modules\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.698825 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.698720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b21010b5-5405-4fa0-b2b1-26f8b760c155-proc\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.708993 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.708967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqqtq\" (UniqueName: \"kubernetes.io/projected/b21010b5-5405-4fa0-b2b1-26f8b760c155-kube-api-access-tqqtq\") pod \"perf-node-gather-daemonset-5dccw\" (UID: \"b21010b5-5405-4fa0-b2b1-26f8b760c155\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:43.863835 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:43.863755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:44.036732 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:44.036162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"]
Jul 14 11:53:44.039356 ip-10-0-137-65 kubenswrapper[2563]: W0714 11:53:44.039299    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb21010b5_5405_4fa0_b2b1_26f8b760c155.slice/crio-13112d5c766dc6e74882cd2a0cc400fce49673d1d918c62c561e4fd8ed54750d WatchSource:0}: Error finding container 13112d5c766dc6e74882cd2a0cc400fce49673d1d918c62c561e4fd8ed54750d: Status 404 returned error can't find the container with id 13112d5c766dc6e74882cd2a0cc400fce49673d1d918c62c561e4fd8ed54750d
Jul 14 11:53:44.180792 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:44.180699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw" event={"ID":"b21010b5-5405-4fa0-b2b1-26f8b760c155","Type":"ContainerStarted","Data":"13112d5c766dc6e74882cd2a0cc400fce49673d1d918c62c561e4fd8ed54750d"}
Jul 14 11:53:45.187863 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:45.187823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw" event={"ID":"b21010b5-5405-4fa0-b2b1-26f8b760c155","Type":"ContainerStarted","Data":"599d4a0b633427b8de3c207b01cb3152d5a50349cda383e4ea1c5051997de6ca"}
Jul 14 11:53:45.188357 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:45.187892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:45.208179 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:45.208126    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw" podStartSLOduration=2.20811022 podStartE2EDuration="2.20811022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:53:45.205904088 +0000 UTC m=+2829.788443096" watchObservedRunningTime="2026-07-14 11:53:45.20811022 +0000 UTC m=+2829.790649225"
Jul 14 11:53:47.295157 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:47.295111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-htvd2_247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c/dns/0.log"
Jul 14 11:53:47.352803 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:47.352773    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-htvd2_247fa4f5-0c27-4aa7-ad3f-87d2cc0bc09c/kube-rbac-proxy/0.log"
Jul 14 11:53:47.475392 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:47.475362    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q65sx_0d5a1d51-4bae-4c57-87f8-46ccd09e8d52/dns-node-resolver/0.log"
Jul 14 11:53:48.163743 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:48.163708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zxxqq_2abf4d3e-c2ff-458c-beb2-f45063b7a331/node-ca/0.log"
Jul 14 11:53:49.162919 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:49.162883    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jm7x5_2f8ed243-8f8c-4bd8-9525-5e403805fa17/istio-proxy/0.log"
Jul 14 11:53:49.341950 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:49.341921    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f79597649-t4bls_177c1b47-85e9-4844-9c70-a845b2d429d3/kube-auth-proxy/0.log"
Jul 14 11:53:49.386155 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:49.386121    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/0.log"
Jul 14 11:53:49.424094 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:49.424014    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/1.log"
Jul 14 11:53:50.263674 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:50.263630    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xkclt_832fb706-40df-48ff-b0b9-ff498711dc2a/serve-healthcheck-canary/0.log"
Jul 14 11:53:50.808963 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:50.808933    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-qq2tp_3da78d2b-82fb-4bcf-a7c7-518c8387914a/insights-operator/0.log"
Jul 14 11:53:50.810092 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:50.810072    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-qq2tp_3da78d2b-82fb-4bcf-a7c7-518c8387914a/insights-operator/1.log"
Jul 14 11:53:51.093392 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:51.093318    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zrhmb_33cb34de-9808-4cc6-9f33-9d2006af17d4/kube-rbac-proxy/0.log"
Jul 14 11:53:51.122023 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:51.121995    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zrhmb_33cb34de-9808-4cc6-9f33-9d2006af17d4/exporter/0.log"
Jul 14 11:53:51.149373 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:51.149347    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zrhmb_33cb34de-9808-4cc6-9f33-9d2006af17d4/extractor/0.log"
Jul 14 11:53:51.206580 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:51.206552    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-5dccw"
Jul 14 11:53:53.610946 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:53.610917    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kgxqw_fc032f67-d50d-4625-8ed7-e1c3a6d8ff2e/manager/0.log"
Jul 14 11:53:53.705290 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:53.705253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:53.781648 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:53.781566    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lrbwz_f85bd830-b0f6-4acd-af45-9434c65e1b57/manager/0.log"
Jul 14 11:53:53.806128 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:53.806099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mc56f_0240294b-4df7-474a-a8f3-9c97f5379cb1/konnectivity-agent/0.log" limit=1024
Jul 14 11:53:53.906357 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:53.906333    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ck7lx_2f86a2c8-2bec-470e-9fcf-7fc78a36bdd0/server/0.log"
Jul 14 11:53:53.989563 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:53.989532    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9qchz_a9eefb21-528a-4f63-b078-8f1032c8c5b7/manager/0.log"
Jul 14 11:53:54.089091 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:54.089011    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-rhh5r_44b2011e-a297-4b0a-9c8f-4749b188b83f/manager/0.log"
Jul 14 11:53:54.118560 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:54.118534    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-rhh5r_44b2011e-a297-4b0a-9c8f-4749b188b83f/copy-manifests/0.log"
Jul 14 11:53:54.294985 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:54.294957    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jm7x5_2f8ed243-8f8c-4bd8-9525-5e403805fa17/istio-proxy/0.log"
Jul 14 11:53:54.428712 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:54.428629    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f79597649-t4bls_177c1b47-85e9-4844-9c70-a845b2d429d3/kube-auth-proxy/0.log"
Jul 14 11:53:54.449977 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:54.449952    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/0.log"
Jul 14 11:53:54.495095 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:54.495057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/1.log"
Jul 14 11:53:55.802783 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:55.802752    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jm7x5_2f8ed243-8f8c-4bd8-9525-5e403805fa17/istio-proxy/0.log"
Jul 14 11:53:55.946608 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:55.946582    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f79597649-t4bls_177c1b47-85e9-4844-9c70-a845b2d429d3/kube-auth-proxy/0.log"
Jul 14 11:53:55.980365 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:55.980331    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/0.log"
Jul 14 11:53:56.023559 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:53:56.023534    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kdhgg_60afec14-4153-48ac-83e0-7d1cf921bab5/istio-proxy/1.log"
Jul 14 11:54:00.884311 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:54:00.884277    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-cnggk_5845ce2e-c307-4065-9a4e-78370d482b09/kube-storage-version-migrator-operator/1.log"
Jul 14 11:54:00.886280 ip-10-0-137-65 kubenswrapper[2563]: I0714 11:54:00.886256    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-cnggk_5845ce2e-c307-4065-9a4e-78370d482b09/kube-storage-version-migrator-operator/0.log"