Jul 24 22:50:14.129672 ip-10-0-143-78 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 22:50:14.129688 ip-10-0-143-78 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 22:50:14.129697 ip-10-0-143-78 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 22:50:14.130030 ip-10-0-143-78 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 22:50:24.263307 ip-10-0-143-78 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 22:50:24.263327 ip-10-0-143-78 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 623bcf38e4f44bf693e8f3bf97fd80f5 --
Jul 24 22:52:17.787025 ip-10-0-143-78 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 22:52:18.163048 ip-10-0-143-78 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 22:52:18.163048 ip-10-0-143-78 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 22:52:18.163048 ip-10-0-143-78 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 22:52:18.163048 ip-10-0-143-78 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 22:52:18.163048 ip-10-0-143-78 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 22:52:18.163980 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.163901    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 22:52:18.169230 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169206    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:18.169230 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169225    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:18.169230 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169231    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:18.169230 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169234    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:18.169230 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169238    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169242    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169246    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169250    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169253    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169257    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169261    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169265    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169269    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169273    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169276    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169280    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169284    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169287    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169292    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169295    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169299    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169303    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169307    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169311    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:18.169518 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169314    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169318    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169322    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169326    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169330    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169338    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169344    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169348    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169355    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169361    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169366    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169371    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169376    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169380    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169385    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169389    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169393    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169397    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169402    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:18.170331 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169406    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169410    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169415    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169419    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169423    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169428    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169432    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169436    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169440    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169444    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169449    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169453    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169458    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169462    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169467    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169472    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169476    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169480    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169485    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169489    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:18.171163 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169493    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169497    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169502    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169506    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169510    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169514    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169518    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169524    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169528    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169532    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169537    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169541    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169545    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169549    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169553    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169559    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169564    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169568    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169572    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169577    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:18.172048 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169582    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169586    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169591    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169595    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169599    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169604    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169610    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169631    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169636    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169640    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169644    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169648    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169653    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169657    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169661    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.169667    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170838    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170849    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170855    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:18.172925 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170860    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170864    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170868    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170873    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170877    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170882    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170886    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170890    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170895    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170899    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170904    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170908    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170912    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170918    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170924    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170928    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170932    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170936    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170941    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170945    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:18.173730 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170950    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170954    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170958    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170963    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170970    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170976    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170981    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170986    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170990    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170995    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.170999    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171004    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171008    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171013    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171017    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171022    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171026    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171030    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171034    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171038    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:18.174330 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171042    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171047    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171053    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171057    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171062    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171068    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171072    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171077    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171081    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171085    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171089    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171094    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171098    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171102    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171107    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171111    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171116    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171120    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171127    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171132    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:18.174917 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171136    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171141    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171145    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171149    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171154    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171158    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171162    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171166    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171170    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171174    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171178    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171184    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171190    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171196    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171201    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171206    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171211    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171215    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171220    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:18.175631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171224    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171230    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171234    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171238    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171243    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171248    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171252    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171256    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171261    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171265    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171269    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171273    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171277    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171281    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171286    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171290    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171294    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171888    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171897    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:18.176122 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171901    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171907    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171913    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171918    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171922    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171926    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171931    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171935    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171939    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171943    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171948    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171952    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171956    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171960    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171966    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171970    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171975    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171980    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171985    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171989    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:18.176694 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171994    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.171998    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172002    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172007    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172011    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172015    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172020    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172024    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172031    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172037    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172042    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172046    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172050    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172054    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172058    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172061    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172065    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172071    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172075    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:18.177248 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172080    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172084    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172089    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172094    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172098    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172103    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172107    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172111    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172116    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172121    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172125    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172129    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172134    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172138    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172142    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172146    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172150    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172154    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172159    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:18.177750 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172163    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172167    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172171    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172175    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172180    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172184    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172188    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172192    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172196    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172200    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172204    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172209    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172214    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172218    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172224    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172229    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172233    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172237    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172241    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172246    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:18.178241 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172250    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172254    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172260    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172265    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172269    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172273    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172277    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172281    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172285    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172290    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172294    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172298    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172302    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172306    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172310    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172315    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172319    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172323    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.172329    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172426    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172442    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 22:52:18.178724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172452    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172458    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172465    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172470    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172476    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172482    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172487    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172493    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172498    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172503    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172508    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172513    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172517    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172522    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172531    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172537    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172542    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172547    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172552    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172557    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172562    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172568    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172573    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 22:52:18.179299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172578    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172583    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172588    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172593    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172597    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172602    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172607    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172613    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172618    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172623    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172627    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172642    2564 flags.go:64] FLAG: --enable-server="true"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172647    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172654    2564 flags.go:64] FLAG: --event-burst="100"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172659    2564 flags.go:64] FLAG: --event-qps="50"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172664    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172669    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172674    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172680    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172685    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172690    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172695    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172700    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172707    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172721    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 22:52:18.179915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172726    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172730    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172735    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172739    2564 flags.go:64] FLAG: --feature-gates=""
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172751    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172756    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172776    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172781    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172786    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172791    2564 flags.go:64] FLAG: --help="false"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172795    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-143-78.ec2.internal"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172800    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172805    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172809    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172815    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172821    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172825    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172830    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172836    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172840    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172845    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172851    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172856    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172860    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 22:52:18.180566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172865    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172870    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172874    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172879    2564 flags.go:64] FLAG: --lock-file=""
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172883    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172888    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172895    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172904    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172910    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172914    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172919    2564 flags.go:64] FLAG: --logging-format="text"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172923    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172928    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172932    2564 flags.go:64] FLAG: --manifest-url=""
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172936    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172943    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172947    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172953    2564 flags.go:64] FLAG: --max-pods="110"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172958    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172964    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172969    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172974    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172979    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172983    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172988    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 22:52:18.181188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.172999    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173006    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173011    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173015    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173020    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173028    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173033    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173038    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173043    2564 flags.go:64] FLAG: --port="10250"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173048    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173052    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00fdd092daeaa5e3b"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173057    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173062    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173067    2564 flags.go:64] FLAG: --register-node="true"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173073    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173078    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173084    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173089    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173093    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173099    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173104    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173109    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173114    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173118    2564 flags.go:64] FLAG: --runonce="false"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173123    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 22:52:18.181817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173128    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173133    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173137    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173142    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173147    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173152    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173157    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173161    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173168    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173173    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173177    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173182    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173187    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173191    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173201    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173205    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173209    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173216    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173221    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173225    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173230    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173236    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173241    2564 flags.go:64] FLAG: --v="2"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173246    2564 flags.go:64] FLAG: --version="false"
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173253    2564 flags.go:64] FLAG: --vmodule=""
Jul 24 22:52:18.182408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173259    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.173265    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173435    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173442    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173446    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173451    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173455    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173460    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173464    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173468    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173473    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173477    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173481    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173485    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173489    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173495    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173499    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173504    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173508    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173512    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:18.183043 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173517    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173521    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173525    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173529    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173533    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173538    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173542    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173546    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173552    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173556    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173561    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173565    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173571    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173575    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173579    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173583    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173588    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173592    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173596    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173600    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:18.183530 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173605    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173609    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173613    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173617    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173621    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173625    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173630    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173635    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173639    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173644    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173648    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173652    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173657    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173661    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173665    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173669    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173673    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173678    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173682    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:18.184070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173688    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173696    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173701    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173708    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173714    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173719    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173724    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173728    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173732    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173736    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173741    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173744    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173748    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173752    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173756    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173776    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173781    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173785    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173789    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173793    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:18.184522 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173799    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173803    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173808    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173812    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173816    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173821    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173825    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173829    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173836    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173841    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173846    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173850    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173855    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173861    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173865    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173870    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173874    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173878    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173882    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:18.185039 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173887    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173892    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.173897    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.174488    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.180853    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.180868    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.180886    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.180893    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180943    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180948    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180951    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180954    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180956    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180959    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180962    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:18.185504 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180964    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180968    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180970    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180973    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180976    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180978    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180981    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180983    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180986    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180988    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180991    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180994    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180997    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.180999    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181002    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181004    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181007    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181009    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181012    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181014    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:18.185894 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181017    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181019    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181023    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181026    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181029    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181031    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181034    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181037    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181040    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181042    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181045    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181047    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181050    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181052    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181055    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181058    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181060    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181062    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181065    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181068    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:18.186372 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181070    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181073    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181075    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181078    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181080    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181084    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181088    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181091    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181094    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181096    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181099    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181102    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181104    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181107    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181111    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181114    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181117    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181119    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181122    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181125    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:18.186869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181127    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181130    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181132    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181135    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181137    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181140    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181142    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181146    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181150    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181153    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181156    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181158    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181161    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181164    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181167    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181170    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181172    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181175    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181177    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:18.187384 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181180    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181183    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181185    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181188    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181190    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181193    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181196    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181199    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181202    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181205    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181209    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181212    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181215    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.181220    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 22:52:18.187869 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181325    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181329    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181332    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181335    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181338    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181341    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181344    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181347    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181350    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181352    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181355    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181357    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181360    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181363    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181365    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181368    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181371    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181373    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181376    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181378    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:18.188216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181381    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181383    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181386    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181388    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181391    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181393    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181397    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181400    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181403    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181406    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181409    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181411    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181414    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181416    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181419    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181421    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181424    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181428    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181432    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181435    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:18.189049 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181437    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181440    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181443    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181446    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181448    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181451    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181453    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181456    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181458    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181461    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181487    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181491    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181494    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181497    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181499    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181527    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181530    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181533    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181536    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181540    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:18.189749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181544    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181548    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181551    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181553    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181557    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181561    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181564    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181566    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181569    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181572    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181575    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181578    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181580    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181583    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181585    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181587    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181590    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181593    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181595    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:18.190397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181598    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181600    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181603    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181605    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181607    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181610    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181612    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181615    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181618    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181621    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181623    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181626    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181629    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181631    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181634    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181637    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181639    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181642    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181645    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:18.190868 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:18.181647    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.181651    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.181740    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.183550    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.184751    2564 server.go:1019] "Starting client certificate rotation"
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.184859    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.184887    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 22:52:18.191363 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.187029    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 22:52:18.191723 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.191704    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 22:52:18.204551 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.204535    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 24 22:52:18.206322 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.206306    2564 log.go:25] "Validated CRI v1 image API"
Jul 24 22:52:18.207166 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.207152    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 22:52:18.210188 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.210168    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8a4f0bdc-cfc2-4e5d-a961-b1e827849b07:/dev/nvme0n1p3 f37357b3-9095-4c7c-8953-a7b121136baf:/dev/nvme0n1p4]
Jul 24 22:52:18.210256 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.210187    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 22:52:18.213318 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.213299    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 22:52:18.215097 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.214989    2564 manager.go:216] Machine: {Timestamp:2026-07-24 22:52:18.213674601 +0000 UTC m=+0.319362536 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099136 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2381f460fa581de6f45ac7dcf33a24 SystemUUID:ec2381f4-60fa-581d-e6f4-5ac7dcf33a24 BootID:623bcf38-e4f4-4bf6-93e8-f3bf97fd80f5 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:18:19:14:1a:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:18:19:14:1a:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:88:fe:0c:b2:52 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 22:52:18.215097 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.215092    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 22:52:18.215199 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.215165    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 22:52:18.217615 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.217590    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 22:52:18.217754 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.217617    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-78.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 22:52:18.218677 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.218666    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 22:52:18.218717 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.218679    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 22:52:18.218717 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.218686    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 22:52:18.219354 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.219344    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 22:52:18.220088 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.220078    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 22:52:18.221689 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221678    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 22:52:18.221822 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221812    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 22:52:18.221863 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221841    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 22:52:18.221863 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221860    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 22:52:18.221934 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221870    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 22:52:18.221934 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221878    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 24 22:52:18.221934 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.221886    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 22:52:18.222293 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.222281    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 22:52:18.222338 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.222307    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 22:52:18.224450 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.224417    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 22:52:18.226658 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.226620    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 22:52:18.226752 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:18.226669    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 22:52:19.481185 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.480988    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 22:52:19.481281 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481269    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 22:52:19.481347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481285    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 22:52:19.481347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481295    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 22:52:19.481347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481309    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 22:52:19.481347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481319    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 22:52:19.481347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481332    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 22:52:19.481347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481340    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 22:52:19.481542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481350    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 22:52:19.481542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481359    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 22:52:19.481542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481381    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 22:52:19.481542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.481390    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 22:52:19.482203 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.482192    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 22:52:19.482254 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.482205    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 22:52:19.484909 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.484886    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 22:52:19.484999 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.484938    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-78.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 22:52:19.485069 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.484982    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 22:52:19.485319 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.485305    2564 server.go:1268] "Started kubelet"
Jul 24 22:52:19.485400 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.485374    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 22:52:19.485444 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.485378    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 22:52:19.485444 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.485435    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 22:52:19.486171 ip-10-0-143-78 systemd[1]: Started Kubernetes Kubelet.
Jul 24 22:52:19.487347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.487333    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 22:52:19.488323 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.488306    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 22:52:19.489250 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.489222    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 22:52:19.491906 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.491009    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-78.ec2.internal.18c55b98e0b36586  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-78.ec2.internal,UID:ip-10-0-143-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-78.ec2.internal,},FirstTimestamp:2026-07-24 22:52:19.485279622 +0000 UTC m=+0.338606154,LastTimestamp:2026-07-24 22:52:19.485279622 +0000 UTC m=+0.338606154,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-78.ec2.internal,}"
Jul 24 22:52:19.493281 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.493263    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 22:52:19.493281 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.493276    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 22:52:19.493563 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.493543    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:19.494282 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494264    2564 factory.go:55] Registering systemd factory
Jul 24 22:52:19.494370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494286    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 24 22:52:19.494370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494302    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 22:52:19.494370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494327    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 22:52:19.494370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494269    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 22:52:19.494534 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494419    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 22:52:19.494534 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494428    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 22:52:19.494534 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494487    2564 factory.go:153] Registering CRI-O factory
Jul 24 22:52:19.494534 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494498    2564 factory.go:223] Registration of the crio container factory successfully
Jul 24 22:52:19.494682 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494543    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 22:52:19.494682 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494563    2564 factory.go:103] Registering Raw factory
Jul 24 22:52:19.494682 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.494581    2564 manager.go:1196] Started watching for new ooms in manager
Jul 24 22:52:19.495037 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.495023    2564 manager.go:349] Starting recovery of all containers
Jul 24 22:52:19.501522 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.501406    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 22:52:19.501522 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.501471    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 22:52:19.507243 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.507226    2564 manager.go:354] Recovery completed
Jul 24 22:52:19.512474 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.512459    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.514072 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514056    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.514145 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514085    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.514145 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514094    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.514581 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514568    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 22:52:19.514581 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514580    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.514664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514580    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 22:52:19.514664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.514644    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 22:52:19.515235 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.515220    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.515294 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.515247    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.515294 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.515259    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.516046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.516035    2564 policy_none.go:49] "None policy: Start"
Jul 24 22:52:19.516096 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.516049    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 22:52:19.516096 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.516059    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 22:52:19.516186 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.516036    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-78.ec2.internal.18c55b98e26abfbc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-78.ec2.internal,UID:ip-10-0-143-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-78.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-78.ec2.internal,},FirstTimestamp:2026-07-24 22:52:19.51407302 +0000 UTC m=+0.367399554,LastTimestamp:2026-07-24 22:52:19.51407302 +0000 UTC m=+0.367399554,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-78.ec2.internal,}"
Jul 24 22:52:19.516936 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.516925    2564 policy_none.go:47] "Start"
Jul 24 22:52:19.517569 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.517551    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-57j4m"
Jul 24 22:52:19.523769 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.523749    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-57j4m"
Jul 24 22:52:19.525846 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.525789    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-78.ec2.internal.18c55b98e26b0091  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-78.ec2.internal,UID:ip-10-0-143-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-78.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-78.ec2.internal,},FirstTimestamp:2026-07-24 22:52:19.514089617 +0000 UTC m=+0.367416147,LastTimestamp:2026-07-24 22:52:19.514089617 +0000 UTC m=+0.367416147,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-78.ec2.internal,}"
Jul 24 22:52:19.548645 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.548633    2564 manager.go:337] "Starting Device Plugin manager"
Jul 24 22:52:19.548715 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.548660    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 22:52:19.548715 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.548669    2564 server.go:87] "Starting device plugin registration server"
Jul 24 22:52:19.548892 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.548880    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 22:52:19.548941 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.548895    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 22:52:19.549004 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.548981    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 22:52:19.549101 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.549084    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 22:52:19.549101 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.549097    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 22:52:19.549556 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.549534    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 22:52:19.549634 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.549587    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:19.649359 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.649335    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.651256 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.650440    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.651256 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.650473    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.651256 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.650505    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.651256 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.650547    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.660212 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.660196    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.660284 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.660217    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-78.ec2.internal\": node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:19.685873 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.685825    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:19.686617 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.686594    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 22:52:19.687831 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.687812    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 22:52:19.687912 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.687850    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 22:52:19.687912 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.687877    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 22:52:19.687985 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.687919    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 22:52:19.689951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.689935    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:19.786365 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.786339    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:19.788493 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.788475    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"]
Jul 24 22:52:19.788543 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.788532    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.790003 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.789988    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.790085 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.790017    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.790085 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.790027    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.792245 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792233    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.792358 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792345    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.792401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792377    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.792954 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792931    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.793044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792961    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.793044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792976    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.793044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.792990    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.793044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.793014    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.793044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.793026    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.795145 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.795233 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795155    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:19.795320 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62d620f13b2ef8b28e0eac26122212df-config\") pod \"kube-apiserver-proxy-ip-10-0-143-78.ec2.internal\" (UID: \"62d620f13b2ef8b28e0eac26122212df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.795377 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795327    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d3f9e4afd66de8e2912992312cf4778-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal\" (UID: \"1d3f9e4afd66de8e2912992312cf4778\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.795377 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d3f9e4afd66de8e2912992312cf4778-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal\" (UID: \"1d3f9e4afd66de8e2912992312cf4778\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.795878 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795863    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:19.795943 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795883    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:19.795943 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.795892    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:19.817765 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.817747    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-78.ec2.internal\" not found" node="ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.822023 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.822006    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-78.ec2.internal\" not found" node="ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.886979 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.886959    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:19.896288 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.896267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62d620f13b2ef8b28e0eac26122212df-config\") pod \"kube-apiserver-proxy-ip-10-0-143-78.ec2.internal\" (UID: \"62d620f13b2ef8b28e0eac26122212df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.896378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.896292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d3f9e4afd66de8e2912992312cf4778-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal\" (UID: \"1d3f9e4afd66de8e2912992312cf4778\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.896378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.896314    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d3f9e4afd66de8e2912992312cf4778-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal\" (UID: \"1d3f9e4afd66de8e2912992312cf4778\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.896378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.896338    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d3f9e4afd66de8e2912992312cf4778-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal\" (UID: \"1d3f9e4afd66de8e2912992312cf4778\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.896378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.896367    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62d620f13b2ef8b28e0eac26122212df-config\") pod \"kube-apiserver-proxy-ip-10-0-143-78.ec2.internal\" (UID: \"62d620f13b2ef8b28e0eac26122212df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.896503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:19.896380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d3f9e4afd66de8e2912992312cf4778-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal\" (UID: \"1d3f9e4afd66de8e2912992312cf4778\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:19.987637 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:19.987597    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:20.088159 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:20.088132    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:20.120632 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.120612    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:20.125199 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.125084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:20.188864 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:20.188843    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:20.289443 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:20.289388    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:20.389945 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:20.389921    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:20.436554 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.436536    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 22:52:20.437044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.436674    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 22:52:20.490296 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:20.490274    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-78.ec2.internal\" not found"
Jul 24 22:52:20.527487 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.527444    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 22:47:19 +0000 UTC" deadline="2028-02-27 07:25:12.596752473 +0000 UTC"
Jul 24 22:52:20.527487 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.527486    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13976h32m52.069270697s"
Jul 24 22:52:20.552640 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.552597    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:20.593991 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.593974    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:20.610263 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.610243    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 22:52:20.612184 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.612163    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal"
Jul 24 22:52:20.620929 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.620910    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 22:52:20.722238 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:20.722205    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3f9e4afd66de8e2912992312cf4778.slice/crio-7720ba630e4c82b14f673bc0c25d0299e8172752021d0d748d3f7f16bee44a3e WatchSource:0}: Error finding container 7720ba630e4c82b14f673bc0c25d0299e8172752021d0d748d3f7f16bee44a3e: Status 404 returned error can't find the container with id 7720ba630e4c82b14f673bc0c25d0299e8172752021d0d748d3f7f16bee44a3e
Jul 24 22:52:20.722443 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:20.722424    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d620f13b2ef8b28e0eac26122212df.slice/crio-acec3efcefba4175a95e525cc971aac1d1472e71e6ad0a50b19112cf121ab94e WatchSource:0}: Error finding container acec3efcefba4175a95e525cc971aac1d1472e71e6ad0a50b19112cf121ab94e: Status 404 returned error can't find the container with id acec3efcefba4175a95e525cc971aac1d1472e71e6ad0a50b19112cf121ab94e
Jul 24 22:52:20.727204 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.727190    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 22:52:20.973945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:20.973875    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:21.022272 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.022250    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:21.475219 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.475172    2564 apiserver.go:52] "Watching apiserver"
Jul 24 22:52:21.483027 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.483000    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 22:52:21.483463 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.483439    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal","openshift-multus/network-metrics-daemon-fjkbr","openshift-network-operator/iptables-alerter-lx2ms","kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8227f","openshift-dns/node-resolver-9vcd8","openshift-image-registry/node-ca-cn5mp","openshift-multus/multus-additional-cni-plugins-nvc8q","openshift-multus/multus-h2sgn","openshift-network-diagnostics/network-check-target-jlnvw","openshift-ovn-kubernetes/ovnkube-node-qcdfg","kube-system/global-pull-secret-syncer-4mrph","kube-system/konnectivity-agent-68qcj"]
Jul 24 22:52:21.485922 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.485902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.488085 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.488065    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.488238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.488218    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.488303 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.488221    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.488419 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.488353    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jpz2p\""
Jul 24 22:52:21.490265 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.490243    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 22:52:21.490380 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.490359    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 22:52:21.490498 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.490480    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 22:52:21.491259 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.490748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:21.491259 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.490814    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:21.491259 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.490826    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.491259 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.490831    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 22:52:21.491259 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.491225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.491527 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.491292    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t868z\""
Jul 24 22:52:21.493014 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.492989    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.494037 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.494019    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 22:52:21.495088 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.495069    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qvd4w\""
Jul 24 22:52:21.495228 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.495138    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.495228 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.495178    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 22:52:21.495359 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.495265    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.496049 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.496031    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.498318 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.498294    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 22:52:21.498318 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.498325    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8wdlh\""
Jul 24 22:52:21.498504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.498362    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.498579 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.498526    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 22:52:21.500549 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.500509    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q497r\""
Jul 24 22:52:21.500640 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.500550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.500757 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.500739    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.500835 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.500786    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.503025 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-crqj6\""
Jul 24 22:52:21.503146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503049    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 22:52:21.503146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503077    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.503146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503057    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 22:52:21.503385 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.503472 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503384    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 22:52:21.503472 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503436    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 22:52:21.503472 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.503360    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.504086 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504065    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-env-overrides\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0614a849-84b5-44da-b548-40e21080bfd7-konnectivity-ca\") pod \"konnectivity-agent-68qcj\" (UID: \"0614a849-84b5-44da-b548-40e21080bfd7\") " pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.504195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-node-log\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:21.504195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-host-slash\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-kubelet\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504231    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-run-netns\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-run-ovn-kubernetes\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-ovnkube-config\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504356    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-slash\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/913a538e-d13c-40f7-b028-51025bc95410-ovn-node-metrics-cert\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504426    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-iptables-alerter-script\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0614a849-84b5-44da-b548-40e21080bfd7-agent-certs\") pod \"konnectivity-agent-68qcj\" (UID: \"0614a849-84b5-44da-b548-40e21080bfd7\") " pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-var-lib-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-cni-bin\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-cni-netd\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcn6j\" (UniqueName: \"kubernetes.io/projected/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-kube-api-access-pcn6j\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504619    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02648613-32e6-4cc7-aaab-e39b8876124d-tmp-dir\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4l8d\" (UniqueName: \"kubernetes.io/projected/02648613-32e6-4cc7-aaab-e39b8876124d-kube-api-access-b4l8d\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-systemd-units\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-systemd\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bdf\" (UniqueName: \"kubernetes.io/projected/913a538e-d13c-40f7-b028-51025bc95410-kube-api-access-p9bdf\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504777    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lft4\" (UniqueName: \"kubernetes.io/projected/f320251f-e6c0-482a-b868-2041a175a057-kube-api-access-4lft4\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-etc-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.504842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504834    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.505539 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-ovn\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.505539 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504896    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02648613-32e6-4cc7-aaab-e39b8876124d-hosts-file\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.505539 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504918    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-ovnkube-script-lib\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.505539 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.504945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-log-socket\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.505785 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.505624    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.505785 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.505768    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 22:52:21.505879 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.505794    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.505879 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.505849    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.506163 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.506044    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7mtcd\""
Jul 24 22:52:21.508144 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.508107    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 22:52:21.508274 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.508260    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 22:52:21.508371 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.508279    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 22:52:21.508433 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.508288    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wqvwk\""
Jul 24 22:52:21.508797 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.508769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.510790 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.510757    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6nv4h\""
Jul 24 22:52:21.510902 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.510787    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 22:52:21.511083 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.511054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.511479 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.511448    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:21.514753 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.514696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:21.514854 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.514761    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:21.524913 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.524896    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l4xqz"
Jul 24 22:52:21.532749 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.532727    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l4xqz"
Jul 24 22:52:21.595381 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.595358    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 22:52:21.605998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.605964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.606139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-ovnkube-script-lib\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.606139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-cni-bin\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.606139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-log-socket\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.606139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-env-overrides\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.606139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0614a849-84b5-44da-b548-40e21080bfd7-konnectivity-ca\") pod \"konnectivity-agent-68qcj\" (UID: \"0614a849-84b5-44da-b548-40e21080bfd7\") " pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.606393 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-registration-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.606393 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-log-socket\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.606393 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj8h2\" (UniqueName: \"kubernetes.io/projected/2436b9ca-9a5e-4194-b48b-8125adc5bf06-kube-api-access-gj8h2\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.606393 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-socket-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.606393 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/011616e0-ac21-4ad2-979f-baaed2fd1174-host\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-kubelet\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-system-cni-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e5143b3-55f1-4714-8a04-a2d8f581b307-dbus\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-sys-fs\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-tuned\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2436b9ca-9a5e-4194-b48b-8125adc5bf06-tmp\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vj5x\" (UniqueName: \"kubernetes.io/projected/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-kube-api-access-8vj5x\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606573    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-env-overrides\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.606629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-iptables-alerter-script\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-cni-binary-copy\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-ovnkube-script-lib\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-kubelet\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606692    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0614a849-84b5-44da-b548-40e21080bfd7-konnectivity-ca\") pod \"konnectivity-agent-68qcj\" (UID: \"0614a849-84b5-44da-b548-40e21080bfd7\") " pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-modprobe-d\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysctl-conf\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-cni-netd\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02648613-32e6-4cc7-aaab-e39b8876124d-tmp-dir\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4l8d\" (UniqueName: \"kubernetes.io/projected/02648613-32e6-4cc7-aaab-e39b8876124d-kube-api-access-b4l8d\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-cni-netd\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606919    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-netns\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-etc-kubernetes\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-device-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.606997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:21.607028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607023    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-systemd\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lft4\" (UniqueName: \"kubernetes.io/projected/f320251f-e6c0-482a-b868-2041a175a057-kube-api-access-4lft4\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-multus-certs\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607101    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e5143b3-55f1-4714-8a04-a2d8f581b307-kubelet-config\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607144    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysctl-d\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607169    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02648613-32e6-4cc7-aaab-e39b8876124d-tmp-dir\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-systemd\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-iptables-alerter-script\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-run\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-host\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-etc-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607298    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02648613-32e6-4cc7-aaab-e39b8876124d-hosts-file\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607324    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-cnibin\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607379    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-etc-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ps95\" (UniqueName: \"kubernetes.io/projected/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-kube-api-access-5ps95\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02648613-32e6-4cc7-aaab-e39b8876124d-hosts-file\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607473    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/011616e0-ac21-4ad2-979f-baaed2fd1174-serviceca\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.607829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-kubernetes\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-systemd\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-kubelet\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-conf-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607570    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-daemon-config\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhn9x\" (UniqueName: \"kubernetes.io/projected/011616e0-ac21-4ad2-979f-baaed2fd1174-kube-api-access-hhn9x\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607625    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-system-cni-dir\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-cnibin\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-cni-multus\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g44jn\" (UniqueName: \"kubernetes.io/projected/7275e19b-9824-496c-ab8f-1268d2845102-kube-api-access-g44jn\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607798    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-node-log\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607835    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-node-log\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-host-slash\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.608624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-run-netns\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-run-ovn-kubernetes\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-host-slash\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.607976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-ovnkube-config\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.608024    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.608106    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:22.108071003 +0000 UTC m=+2.961397525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-run-netns\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-run-ovn-kubernetes\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-os-release\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/913a538e-d13c-40f7-b028-51025bc95410-ovnkube-config\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-hostroot\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-slash\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/913a538e-d13c-40f7-b028-51025bc95410-ovn-node-metrics-cert\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0614a849-84b5-44da-b548-40e21080bfd7-agent-certs\") pod \"konnectivity-agent-68qcj\" (UID: \"0614a849-84b5-44da-b548-40e21080bfd7\") " pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608570    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-socket-dir-parent\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-k8s-cni-cncf-io\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.609401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-var-lib-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-cni-bin\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcn6j\" (UniqueName: \"kubernetes.io/projected/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-kube-api-access-pcn6j\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-systemd-units\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bdf\" (UniqueName: \"kubernetes.io/projected/913a538e-d13c-40f7-b028-51025bc95410-kube-api-access-p9bdf\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-lib-modules\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608823    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-var-lib-kubelet\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-cni-binary-copy\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-ovn\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-cni-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608936    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-os-release\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysconfig\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608957    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.608979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-sys\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.610267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609101    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-var-lib-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-cni-bin\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609322    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-host-slash\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-openvswitch\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-systemd-units\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.610924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.609488    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/913a538e-d13c-40f7-b028-51025bc95410-run-ovn\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.613568 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.613477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/913a538e-d13c-40f7-b028-51025bc95410-ovn-node-metrics-cert\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.614807 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.614407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0614a849-84b5-44da-b548-40e21080bfd7-agent-certs\") pod \"konnectivity-agent-68qcj\" (UID: \"0614a849-84b5-44da-b548-40e21080bfd7\") " pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.616199 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.616071    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lft4\" (UniqueName: \"kubernetes.io/projected/f320251f-e6c0-482a-b868-2041a175a057-kube-api-access-4lft4\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:21.616306 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.616227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4l8d\" (UniqueName: \"kubernetes.io/projected/02648613-32e6-4cc7-aaab-e39b8876124d-kube-api-access-b4l8d\") pod \"node-resolver-9vcd8\" (UID: \"02648613-32e6-4cc7-aaab-e39b8876124d\") " pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.618049 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.617778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcn6j\" (UniqueName: \"kubernetes.io/projected/477b1d48-f7e2-47d5-81f1-3fa64bcd0606-kube-api-access-pcn6j\") pod \"iptables-alerter-lx2ms\" (UID: \"477b1d48-f7e2-47d5-81f1-3fa64bcd0606\") " pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.618462 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.618443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bdf\" (UniqueName: \"kubernetes.io/projected/913a538e-d13c-40f7-b028-51025bc95410-kube-api-access-p9bdf\") pod \"ovnkube-node-qcdfg\" (UID: \"913a538e-d13c-40f7-b028-51025bc95410\") " pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.693899 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.693844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal" event={"ID":"62d620f13b2ef8b28e0eac26122212df","Type":"ContainerStarted","Data":"acec3efcefba4175a95e525cc971aac1d1472e71e6ad0a50b19112cf121ab94e"}
Jul 24 22:52:21.695740 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.695713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal" event={"ID":"1d3f9e4afd66de8e2912992312cf4778","Type":"ContainerStarted","Data":"7720ba630e4c82b14f673bc0c25d0299e8172752021d0d748d3f7f16bee44a3e"}
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709576    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-system-cni-dir\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-cnibin\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-cni-multus\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44jn\" (UniqueName: \"kubernetes.io/projected/7275e19b-9824-496c-ab8f-1268d2845102-kube-api-access-g44jn\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-os-release\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-hostroot\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-socket-dir-parent\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-k8s-cni-cncf-io\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709950    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-lib-modules\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-var-lib-kubelet\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-cni-binary-copy\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.711146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-os-release\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710023    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-cni-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-os-release\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710055    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-k8s-cni-cncf-io\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysconfig\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-sys\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-system-cni-dir\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710172    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-sys\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-cnibin\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710215    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-cni-bin\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7275e19b-9824-496c-ab8f-1268d2845102-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-lib-modules\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-os-release\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-cni-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysconfig\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-cni-bin\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-registration-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.712050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710706    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj8h2\" (UniqueName: \"kubernetes.io/projected/2436b9ca-9a5e-4194-b48b-8125adc5bf06-kube-api-access-gj8h2\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-socket-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710743    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/011616e0-ac21-4ad2-979f-baaed2fd1174-host\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-system-cni-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e5143b3-55f1-4714-8a04-a2d8f581b307-dbus\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.710818    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710861    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-sys-fs\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.710868    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:22.210852122 +0000 UTC m=+3.064178645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-hostroot\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710921    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-registration-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-socket-dir-parent\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.709838    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-cni-multus\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-var-lib-kubelet\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.710818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-sys-fs\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-tuned\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2436b9ca-9a5e-4194-b48b-8125adc5bf06-tmp\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.712894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vj5x\" (UniqueName: \"kubernetes.io/projected/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-kube-api-access-8vj5x\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-cni-binary-copy\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-socket-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/011616e0-ac21-4ad2-979f-baaed2fd1174-host\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-system-cni-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e5143b3-55f1-4714-8a04-a2d8f581b307-dbus\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-cni-binary-copy\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.711866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7275e19b-9824-496c-ab8f-1268d2845102-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.713751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.713613    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-cni-binary-copy\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-modprobe-d\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysctl-conf\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-netns\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-etc-kubernetes\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-device-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714340    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-modprobe-d\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714379    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-multus-certs\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714385    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-netns\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e5143b3-55f1-4714-8a04-a2d8f581b307-kubelet-config\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysctl-d\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-run\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-host\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-cnibin\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysctl-conf\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.715701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ps95\" (UniqueName: \"kubernetes.io/projected/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-kube-api-access-5ps95\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/011616e0-ac21-4ad2-979f-baaed2fd1174-serviceca\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-kubernetes\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-systemd\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-kubelet\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-conf-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-daemon-config\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhn9x\" (UniqueName: \"kubernetes.io/projected/011616e0-ac21-4ad2-979f-baaed2fd1174-kube-api-access-hhn9x\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/011616e0-ac21-4ad2-979f-baaed2fd1174-serviceca\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.714973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-etc-kubernetes\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-device-dir\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-kubernetes\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-systemd\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-var-lib-kubelet\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715231    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-conf-dir\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e5143b3-55f1-4714-8a04-a2d8f581b307-kubelet-config\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-host-run-multus-certs\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.716587 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-sysctl-d\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.717441 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-run\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.717441 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2436b9ca-9a5e-4194-b48b-8125adc5bf06-host\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.717441 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-cnibin\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.717441 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.715707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-multus-daemon-config\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.717782 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.717762    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2436b9ca-9a5e-4194-b48b-8125adc5bf06-tmp\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.719710 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.719688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2436b9ca-9a5e-4194-b48b-8125adc5bf06-etc-tuned\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.723469 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.723197    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:21.723469 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.723220    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:21.723469 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.723234    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:21.723469 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:21.723325    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:22.223308312 +0000 UTC m=+3.076634832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:21.725399 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.725329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vj5x\" (UniqueName: \"kubernetes.io/projected/f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144-kube-api-access-8vj5x\") pod \"multus-h2sgn\" (UID: \"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144\") " pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.727114 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.727075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj8h2\" (UniqueName: \"kubernetes.io/projected/2436b9ca-9a5e-4194-b48b-8125adc5bf06-kube-api-access-gj8h2\") pod \"tuned-8227f\" (UID: \"2436b9ca-9a5e-4194-b48b-8125adc5bf06\") " pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.727390 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.727371    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ps95\" (UniqueName: \"kubernetes.io/projected/7bb1cda8-cfef-4641-aabc-e576de2bd6a3-kube-api-access-5ps95\") pod \"aws-ebs-csi-driver-node-nvlqg\" (UID: \"7bb1cda8-cfef-4641-aabc-e576de2bd6a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.729050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.729019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhn9x\" (UniqueName: \"kubernetes.io/projected/011616e0-ac21-4ad2-979f-baaed2fd1174-kube-api-access-hhn9x\") pod \"node-ca-cn5mp\" (UID: \"011616e0-ac21-4ad2-979f-baaed2fd1174\") " pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.729735 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.729699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g44jn\" (UniqueName: \"kubernetes.io/projected/7275e19b-9824-496c-ab8f-1268d2845102-kube-api-access-g44jn\") pod \"multus-additional-cni-plugins-nvc8q\" (UID: \"7275e19b-9824-496c-ab8f-1268d2845102\") " pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.797641 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.797610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9vcd8"
Jul 24 22:52:21.804353 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.804175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:21.814739 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.814719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lx2ms"
Jul 24 22:52:21.819367 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.819342    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:21.826103 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.826084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8227f"
Jul 24 22:52:21.832788 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.832769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nvc8q"
Jul 24 22:52:21.840779 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.840764    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg"
Jul 24 22:52:21.846435 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.846416    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cn5mp"
Jul 24 22:52:21.850100 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.850080    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2sgn"
Jul 24 22:52:21.977767 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:21.977730    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0614a849_84b5_44da_b548_40e21080bfd7.slice/crio-6798ffb873255075661f347ffe28125b0b82dc9e575a844c7ce68a5fd8b5233e WatchSource:0}: Error finding container 6798ffb873255075661f347ffe28125b0b82dc9e575a844c7ce68a5fd8b5233e: Status 404 returned error can't find the container with id 6798ffb873255075661f347ffe28125b0b82dc9e575a844c7ce68a5fd8b5233e
Jul 24 22:52:21.978216 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:21.978192    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2436b9ca_9a5e_4194_b48b_8125adc5bf06.slice/crio-5574ed27a6e73028399bce67ac2994ee258776a5181add1e666064dc47252e09 WatchSource:0}: Error finding container 5574ed27a6e73028399bce67ac2994ee258776a5181add1e666064dc47252e09: Status 404 returned error can't find the container with id 5574ed27a6e73028399bce67ac2994ee258776a5181add1e666064dc47252e09
Jul 24 22:52:21.979979 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:21.979950    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ee5df0_ea62_4fe0_a7fc_c3322dc3a144.slice/crio-bd91b0cd820757ae21567c273d5ba2c2dba1bd78136e27c676bc2ac0ae57aee9 WatchSource:0}: Error finding container bd91b0cd820757ae21567c273d5ba2c2dba1bd78136e27c676bc2ac0ae57aee9: Status 404 returned error can't find the container with id bd91b0cd820757ae21567c273d5ba2c2dba1bd78136e27c676bc2ac0ae57aee9
Jul 24 22:52:21.986062 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:21.986034    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477b1d48_f7e2_47d5_81f1_3fa64bcd0606.slice/crio-3d5bb92cf045e379af2819660f11b92f6a1894c021aabe47c2d042a4ae9f3238 WatchSource:0}: Error finding container 3d5bb92cf045e379af2819660f11b92f6a1894c021aabe47c2d042a4ae9f3238: Status 404 returned error can't find the container with id 3d5bb92cf045e379af2819660f11b92f6a1894c021aabe47c2d042a4ae9f3238
Jul 24 22:52:21.986728 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:21.986690    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02648613_32e6_4cc7_aaab_e39b8876124d.slice/crio-9b4bd2b86ab7cf0f90c5a730c7be680019def895bdfbd0aada60a49b0c8ea727 WatchSource:0}: Error finding container 9b4bd2b86ab7cf0f90c5a730c7be680019def895bdfbd0aada60a49b0c8ea727: Status 404 returned error can't find the container with id 9b4bd2b86ab7cf0f90c5a730c7be680019def895bdfbd0aada60a49b0c8ea727
Jul 24 22:52:21.987482 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:21.987456    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011616e0_ac21_4ad2_979f_baaed2fd1174.slice/crio-81657428b96de4fbef5a597cf428b82dc4ed74b735e1f3ee1a0fbc6dd2b8ad0e WatchSource:0}: Error finding container 81657428b96de4fbef5a597cf428b82dc4ed74b735e1f3ee1a0fbc6dd2b8ad0e: Status 404 returned error can't find the container with id 81657428b96de4fbef5a597cf428b82dc4ed74b735e1f3ee1a0fbc6dd2b8ad0e
Jul 24 22:52:21.998471 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:21.998450    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:22.117451 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.117282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:22.117451 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.117434    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:22.117627 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.117504    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:23.117488839 +0000 UTC m=+3.970815357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:22.218155 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.218107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:22.218301 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.218225    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:22.218301 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.218284    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:23.218266281 +0000 UTC m=+4.071592809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:22.318725 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.318695    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:22.318890 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.318871    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:22.318956 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.318895    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:22.318956 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.318909    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:22.319055 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.318975    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:23.318953967 +0000 UTC m=+4.172280499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:22.536370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.536251    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:47:21 +0000 UTC" deadline="2028-05-02 16:07:58.34080051 +0000 UTC"
Jul 24 22:52:22.536370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.536284    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15545h15m35.804520519s"
Jul 24 22:52:22.688134 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.688088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:22.688289 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:22.688236    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:22.702296 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.702218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerStarted","Data":"59b253dad1c2783dfb51e503ed947db4dd6285dfe9c35dad3cd46c0bba2a63a8"}
Jul 24 22:52:22.704230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.704140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg" event={"ID":"7bb1cda8-cfef-4641-aabc-e576de2bd6a3","Type":"ContainerStarted","Data":"91711dd64a1efc7c7ca4e09e9b28c99bd6ffbf8a3e18e13fd954138fa7d9e260"}
Jul 24 22:52:22.707931 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.707908    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9vcd8" event={"ID":"02648613-32e6-4cc7-aaab-e39b8876124d","Type":"ContainerStarted","Data":"9b4bd2b86ab7cf0f90c5a730c7be680019def895bdfbd0aada60a49b0c8ea727"}
Jul 24 22:52:22.710021 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.709998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lx2ms" event={"ID":"477b1d48-f7e2-47d5-81f1-3fa64bcd0606","Type":"ContainerStarted","Data":"3d5bb92cf045e379af2819660f11b92f6a1894c021aabe47c2d042a4ae9f3238"}
Jul 24 22:52:22.711837 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.711807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8227f" event={"ID":"2436b9ca-9a5e-4194-b48b-8125adc5bf06","Type":"ContainerStarted","Data":"5574ed27a6e73028399bce67ac2994ee258776a5181add1e666064dc47252e09"}
Jul 24 22:52:22.713774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.713750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-68qcj" event={"ID":"0614a849-84b5-44da-b548-40e21080bfd7","Type":"ContainerStarted","Data":"6798ffb873255075661f347ffe28125b0b82dc9e575a844c7ce68a5fd8b5233e"}
Jul 24 22:52:22.717290 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.717258    2564 generic.go:356] "Generic (PLEG): container finished" podID="1d3f9e4afd66de8e2912992312cf4778" containerID="04df0b39a950e219a0632fca2b54ecc8dd9f1fb1e305ca8806f46bcd83262783" exitCode=0
Jul 24 22:52:22.717386 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.717339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal" event={"ID":"1d3f9e4afd66de8e2912992312cf4778","Type":"ContainerDied","Data":"04df0b39a950e219a0632fca2b54ecc8dd9f1fb1e305ca8806f46bcd83262783"}
Jul 24 22:52:22.720462 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.720438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"ee9ef240caff0ce69b9770641ea0208ebb32f3d5a0941ce3373ea14443b96286"}
Jul 24 22:52:22.741336 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.741299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cn5mp" event={"ID":"011616e0-ac21-4ad2-979f-baaed2fd1174","Type":"ContainerStarted","Data":"81657428b96de4fbef5a597cf428b82dc4ed74b735e1f3ee1a0fbc6dd2b8ad0e"}
Jul 24 22:52:22.754668 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:22.754635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2sgn" event={"ID":"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144","Type":"ContainerStarted","Data":"bd91b0cd820757ae21567c273d5ba2c2dba1bd78136e27c676bc2ac0ae57aee9"}
Jul 24 22:52:23.126582 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.125939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:23.126582 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.126093    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:23.126582 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.126169    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:25.126148194 +0000 UTC m=+5.979474726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:23.226706 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.226674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:23.226902 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.226888    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:23.226964 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.226945    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:25.226927746 +0000 UTC m=+6.080254288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:23.327506 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.327424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:23.327666 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.327608    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:23.327666 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.327627    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:23.327666 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.327640    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:23.327827 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.327696    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:25.327678755 +0000 UTC m=+6.181005278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:23.536726 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.536641    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:47:21 +0000 UTC" deadline="2028-02-04 16:14:55.234102947 +0000 UTC"
Jul 24 22:52:23.536726 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.536695    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13433h22m31.697411793s"
Jul 24 22:52:23.690840 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.690773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:23.690999 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.690906    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:23.690999 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.690981    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:23.691110 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:23.691054    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:23.770656 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.769998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal" event={"ID":"62d620f13b2ef8b28e0eac26122212df","Type":"ContainerStarted","Data":"1839150a5149669c5ffe19ba940ee08e25aa810ca16f53ca7ce019f56a5d396c"}
Jul 24 22:52:23.778678 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.778178    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal" event={"ID":"1d3f9e4afd66de8e2912992312cf4778","Type":"ContainerStarted","Data":"ebabe1b06a8ecbefc683d8217326e137ec60218f9347f62374c8270e3c45b862"}
Jul 24 22:52:23.806399 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:23.806041    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-78.ec2.internal" podStartSLOduration=3.806023975 podStartE2EDuration="3.806023975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:23.785768411 +0000 UTC m=+4.639094952" watchObservedRunningTime="2026-07-24 22:52:23.806023975 +0000 UTC m=+4.659350517"
Jul 24 22:52:24.688597 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:24.688562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:24.689140 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:24.688699    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:25.143693 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:25.143611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:25.143856 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.143773    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:25.143856 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.143835    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:29.143815291 +0000 UTC m=+9.997141811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:25.245420 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:25.244775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:25.245420 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.244923    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:25.245420 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.244984    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:29.244965926 +0000 UTC m=+10.098292445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:25.345508 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:25.345422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:25.345672 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.345608    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:25.345672 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.345636    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:25.345672 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.345649    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:25.345831 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.345709    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:29.345688642 +0000 UTC m=+10.199015175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:25.693017 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:25.692406    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:25.693017 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.692518    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:25.693017 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:25.692859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:25.693017 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:25.692955    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:26.688997 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:26.688859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:26.688997 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:26.688966    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:27.688889 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:27.688826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:27.688889 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:27.688845    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:27.689385 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:27.688950    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:27.689385 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:27.689097    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:28.688371 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:28.688335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:28.688543 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:28.688478    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:29.174854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:29.174783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:29.175247 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.174945    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:29.175247 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.175014    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:37.174998407 +0000 UTC m=+18.028324929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:29.276080 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:29.275508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:29.276080 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.275699    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:29.276080 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.275758    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:37.275739951 +0000 UTC m=+18.129066472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:29.376503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:29.376399    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:29.376675 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.376594    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:29.376675 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.376618    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:29.376675 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.376629    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:29.376840 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.376688    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:37.376671055 +0000 UTC m=+18.229997582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:29.689481 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:29.689106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:29.689481 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.689230    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:29.689481 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:29.689312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:29.689481 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:29.689405    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:30.689274 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:30.689240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:30.689721 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:30.689388    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:31.689223 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:31.689186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:31.689223 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:31.689217    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:31.689760 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:31.689309    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:31.689760 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:31.689445    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:32.688212 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:32.688134    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:32.688369 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:32.688273    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:33.688839 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:33.688806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:33.689281 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:33.688942    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:33.689281 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:33.689042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:33.689281 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:33.689158    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:34.688425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:34.688399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:34.688594 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:34.688506    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:35.688589 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:35.688556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:35.689045 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:35.688608    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:35.689045 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:35.688706    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:35.689045 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:35.688843    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:36.688559 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:36.688473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:36.688741 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:36.688591    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:37.239774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:37.239740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:37.239929 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.239887    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:37.239970 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.239951    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:53.239933545 +0000 UTC m=+34.093260085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:37.340838 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:37.340810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:37.341026 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.340929    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:37.341026 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.340995    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:53.340974927 +0000 UTC m=+34.194301454 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:37.441269 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:37.441226    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:37.441420 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.441398    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:37.441462 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.441422    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:37.441462 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.441434    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:37.441584 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.441491    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:53.44147157 +0000 UTC m=+34.294798088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:37.688729 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:37.688654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:37.688875 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:37.688665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:37.688875 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.688770    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:37.688875 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:37.688858    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:38.689058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:38.689025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:38.689468 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:38.689156    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:39.690040 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:39.690003    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:39.690523 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:39.690111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:39.690523 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:39.690135    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:39.690523 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:39.690277    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:40.688376 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.688063    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:40.688483 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:40.688462    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:40.813097 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.813066    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"8bf386f7f5d85084c2352fc88de3982634cf8ce936a4a3a812f952f2a16ba40d"}
Jul 24 22:52:40.813845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.813157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"ec7464820693de22f3184d5944b9d16e96b9b33ddbc1f53ec51aa61d4e7bfe06"}
Jul 24 22:52:40.813845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.813236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"296efefd150b9b5c53b66c364d720c6b6d5b4f10a9251b3ad644a35da3620270"}
Jul 24 22:52:40.813845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.813279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"3f9b305e8fd4f3ebc10c79618a44e308bf04cca5250dd47945e71fa8c56ed13b"}
Jul 24 22:52:40.813845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.813324    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"dcd65ad7e631c46916198ce1f5c321c8bff09a024dad64630cc37736fca8556a"}
Jul 24 22:52:40.813845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.813372    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"3b2d1120fdee862f0dafb75f44a5eaea9fddc3f74a2f02290eadb46919059ecc"}
Jul 24 22:52:40.814483 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.814462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cn5mp" event={"ID":"011616e0-ac21-4ad2-979f-baaed2fd1174","Type":"ContainerStarted","Data":"3876ce8bf9e780ec7586a8e17f45797108f841cca50d9e6b201e7d543c313d94"}
Jul 24 22:52:40.815927 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.815903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2sgn" event={"ID":"f4ee5df0-ea62-4fe0-a7fc-c3322dc3a144","Type":"ContainerStarted","Data":"65141490f6350d56aec859c576e693a408b54b47e93b56bc8194e849ab6660c9"}
Jul 24 22:52:40.817563 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.817541    2564 generic.go:356] "Generic (PLEG): container finished" podID="7275e19b-9824-496c-ab8f-1268d2845102" containerID="588214b47d4b72bcbb308e26f74e21b0ad7fedd03ada3c6a3faacd602caed466" exitCode=0
Jul 24 22:52:40.817653 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.817622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerDied","Data":"588214b47d4b72bcbb308e26f74e21b0ad7fedd03ada3c6a3faacd602caed466"}
Jul 24 22:52:40.819267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.819249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg" event={"ID":"7bb1cda8-cfef-4641-aabc-e576de2bd6a3","Type":"ContainerStarted","Data":"ba86b69b7f78e871ba80fff5033d85698026b00958630f36e3f0da1e69c24e7b"}
Jul 24 22:52:40.820910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.820889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9vcd8" event={"ID":"02648613-32e6-4cc7-aaab-e39b8876124d","Type":"ContainerStarted","Data":"f7e126c87037ebd9117b3124758cc037575a1678650e127a988a491872325998"}
Jul 24 22:52:40.822298 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.822280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8227f" event={"ID":"2436b9ca-9a5e-4194-b48b-8125adc5bf06","Type":"ContainerStarted","Data":"09ceb032a0199c435ad57234a01f47bb5d7013576eb1cdd46141e037348c5922"}
Jul 24 22:52:40.824178 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.824113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-68qcj" event={"ID":"0614a849-84b5-44da-b548-40e21080bfd7","Type":"ContainerStarted","Data":"5df171fbc6837a7ed1995ae96ad34be67203b674f3976fbfc22ca0a5272a4464"}
Jul 24 22:52:40.828994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.828938    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cn5mp" podStartSLOduration=7.53267546 podStartE2EDuration="21.828922409s" totalImagesPullingTime="14.296246949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:40.828848391 +0000 UTC m=+21.682174931" watchObservedRunningTime="2026-07-24 22:52:40.828922409 +0000 UTC m=+21.682248950"
Jul 24 22:52:40.829664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.829632    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-78.ec2.internal" podStartSLOduration=20.82962101 podStartE2EDuration="20.82962101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:23.806386451 +0000 UTC m=+4.659712983" watchObservedRunningTime="2026-07-24 22:52:40.82962101 +0000 UTC m=+21.682947549"
Jul 24 22:52:40.844035 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.843995    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8227f" podStartSLOduration=3.578170202 podStartE2EDuration="21.843984821s" totalImagesPullingTime="18.265814619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:40.843557679 +0000 UTC m=+21.696884218" watchObservedRunningTime="2026-07-24 22:52:40.843984821 +0000 UTC m=+21.697311355"
Jul 24 22:52:40.887368 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.887315    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9vcd8" podStartSLOduration=7.5909917700000005 podStartE2EDuration="21.887299951s" totalImagesPullingTime="14.296308181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:40.887153896 +0000 UTC m=+21.740480437" watchObservedRunningTime="2026-07-24 22:52:40.887299951 +0000 UTC m=+21.740626491"
Jul 24 22:52:40.903863 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.903774    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-68qcj" podStartSLOduration=3.651375208 podStartE2EDuration="21.903763499s" totalImagesPullingTime="18.252388291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:40.903350013 +0000 UTC m=+21.756676610" watchObservedRunningTime="2026-07-24 22:52:40.903763499 +0000 UTC m=+21.757090040"
Jul 24 22:52:40.919089 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:40.919044    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h2sgn" podStartSLOduration=3.389604589 podStartE2EDuration="21.919029636s" totalImagesPullingTime="18.529425047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:40.918869718 +0000 UTC m=+21.772196258" watchObservedRunningTime="2026-07-24 22:52:40.919029636 +0000 UTC m=+21.772356175"
Jul 24 22:52:41.688264 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:41.688188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:41.688436 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:41.688311    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:41.688436 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:41.688363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:41.688551 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:41.688487    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:41.827672 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:41.827635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lx2ms" event={"ID":"477b1d48-f7e2-47d5-81f1-3fa64bcd0606","Type":"ContainerStarted","Data":"4d521453662a1ba2448983f69095542a7292ceca577444d404090059467f55c4"}
Jul 24 22:52:41.841716 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:41.841695    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 22:52:41.842073 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:41.842036    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lx2ms" podStartSLOduration=4.60357674 podStartE2EDuration="22.842021427s" totalImagesPullingTime="18.238444687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:41.841671718 +0000 UTC m=+22.694998262" watchObservedRunningTime="2026-07-24 22:52:41.842021427 +0000 UTC m=+22.695347966"
Jul 24 22:52:42.560627 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.560516    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T22:52:41.841711106Z","UUID":"378f5660-3395-468b-954a-b1e570ff7e5b","Handler":null,"Name":"","Endpoint":""}
Jul 24 22:52:42.563038 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.563005    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 22:52:42.563177 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.563059    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 22:52:42.688760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.688585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:42.688912 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:42.688831    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:42.765388 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.765353    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:42.766033 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.766010    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:42.832566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:42.832480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg" event={"ID":"7bb1cda8-cfef-4641-aabc-e576de2bd6a3","Type":"ContainerStarted","Data":"9d76a013e14e0a1f64a1c2d8fa090bcdabe356cd94ecf3119b72f4b52c5bee0d"}
Jul 24 22:52:43.688721 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:43.688698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:43.688721 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:43.688709    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:43.688971 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:43.688814    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:43.689030 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:43.688981    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:43.836674 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:43.836641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg" event={"ID":"7bb1cda8-cfef-4641-aabc-e576de2bd6a3","Type":"ContainerStarted","Data":"dbb287b5245665f801ad902c322f9b6fe4c675523171ca4d0d43e14bda59a844"}
Jul 24 22:52:43.840487 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:43.840429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"4397dc65438fb5c0f775f0d69bd5952fa6982b341ab6be9ddee7d84535ec8797"}
Jul 24 22:52:43.840487 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:43.840457    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 22:52:43.855320 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:43.855270    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nvlqg" podStartSLOduration=4.073047631 podStartE2EDuration="24.855253909s" totalImagesPullingTime="20.782206278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:43.855103073 +0000 UTC m=+24.708429615" watchObservedRunningTime="2026-07-24 22:52:43.855253909 +0000 UTC m=+24.708580445"
Jul 24 22:52:44.688807 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:44.688769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:44.688989 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:44.688909    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:45.688473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.688304    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:45.689056 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.688366    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:45.689056 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:45.688567    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:45.689056 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:45.688619    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:45.850794 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.850763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" event={"ID":"913a538e-d13c-40f7-b028-51025bc95410","Type":"ContainerStarted","Data":"961a8a2896a189943344c84731986bf0c33548c553ceb8d5c72ae83f89ad3ca9"}
Jul 24 22:52:45.851018 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.850995    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:45.852576 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.852555    2564 generic.go:356] "Generic (PLEG): container finished" podID="7275e19b-9824-496c-ab8f-1268d2845102" containerID="4976f2c3889d4e7fbfe26e58fd646dafa8540a81cb9e33aaa69a1ee0e15145df" exitCode=0
Jul 24 22:52:45.852674 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.852589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerDied","Data":"4976f2c3889d4e7fbfe26e58fd646dafa8540a81cb9e33aaa69a1ee0e15145df"}
Jul 24 22:52:45.866378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.866361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:45.875895 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:45.875860    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg" podStartSLOduration=8.608524094 podStartE2EDuration="26.875849599s" totalImagesPullingTime="18.267325505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:45.87477448 +0000 UTC m=+26.728101019" watchObservedRunningTime="2026-07-24 22:52:45.875849599 +0000 UTC m=+26.729176138"
Jul 24 22:52:46.688195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:46.688166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:46.688381 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:46.688263    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:46.855317 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:46.855291    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:46.855693 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:46.855374    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:46.869393 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:46.869370    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:52:47.200065 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.199992    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4mrph"]
Jul 24 22:52:47.200220 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.200098    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:47.200281 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:47.200226    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:47.207287 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.207260    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjkbr"]
Jul 24 22:52:47.207404 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.207387    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:47.207488 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:47.207470    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:47.207916 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.207883    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jlnvw"]
Jul 24 22:52:47.208003 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.207966    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:47.208047 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:47.208036    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:47.859138 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.859093    2564 generic.go:356] "Generic (PLEG): container finished" podID="7275e19b-9824-496c-ab8f-1268d2845102" containerID="031399db5e952ed3867f9b1163f2bfee8f8e46f59b2da2f8a1fb8aa4e07544c3" exitCode=0
Jul 24 22:52:47.859488 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:47.859183    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerDied","Data":"031399db5e952ed3867f9b1163f2bfee8f8e46f59b2da2f8a1fb8aa4e07544c3"}
Jul 24 22:52:48.223853 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.223773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:48.223983 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.223901    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 22:52:48.224312 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.224297    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-68qcj"
Jul 24 22:52:48.688860 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.688834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:48.688931 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.688893    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:48.689020 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:48.688996    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:48.689067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.689007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:48.689112 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:48.689091    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:48.689192 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:48.689177    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:48.863675 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.863642    2564 generic.go:356] "Generic (PLEG): container finished" podID="7275e19b-9824-496c-ab8f-1268d2845102" containerID="331cf3a9100634e6eb5c84fc16093981bae5ed03e794046b089f5b7a8d47c8cc" exitCode=0
Jul 24 22:52:48.864103 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:48.863723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerDied","Data":"331cf3a9100634e6eb5c84fc16093981bae5ed03e794046b089f5b7a8d47c8cc"}
Jul 24 22:52:50.688897 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:50.688659    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:50.689596 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:50.688701    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:50.689596 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:50.688708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:50.689596 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:50.689002    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:50.689596 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:50.689076    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:50.689596 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:50.689172    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:52.688535 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:52.688498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:52.688535 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:52.688520    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:52.688986 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:52.688624    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:52.688986 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:52.688654    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlnvw" podUID="3db13fdd-3021-4d6f-9a4b-68d0abba8f78"
Jul 24 22:52:52.688986 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:52.688765    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjkbr" podUID="f320251f-e6c0-482a-b868-2041a175a057"
Jul 24 22:52:52.688986 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:52.688856    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mrph" podUID="7e5143b3-55f1-4714-8a04-a2d8f581b307"
Jul 24 22:52:53.021066 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.021040    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-78.ec2.internal" event="NodeReady"
Jul 24 22:52:53.021192 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.021186    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 22:52:53.055599 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.055573    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c6bcc8959-5xkt9"]
Jul 24 22:52:53.067299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.067278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.069420 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.069395    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 22:52:53.069546 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.069395    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 22:52:53.069546 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.069404    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tmxk\""
Jul 24 22:52:53.069546 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.069404    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 22:52:53.071170 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.071150    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pqdb6"]
Jul 24 22:52:53.076010 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.075688    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 22:52:53.082485 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.082453    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6bcc8959-5xkt9"]
Jul 24 22:52:53.082650 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.082495    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rpbq7"]
Jul 24 22:52:53.082650 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.082604    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.084739 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.084721    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 22:52:53.085164 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.085147    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j4zlm\""
Jul 24 22:52:53.085264 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.085173    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 22:52:53.091235 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.091214    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rpbq7"]
Jul 24 22:52:53.091316 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.091251    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pqdb6"]
Jul 24 22:52:53.091378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.091357    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.093988 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.093880    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 22:52:53.094095 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.093985    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 22:52:53.094095 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.093998    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 22:52:53.094411 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.094389    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z8ljk\""
Jul 24 22:52:53.165661 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-image-registry-private-configuration\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165768 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-trusted-ca\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165768 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e78f7d-9f01-4fa3-8913-246be64bd119-config-volume\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.165768 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrgsj\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-kube-api-access-lrgsj\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165768 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/707db365-5d14-4f25-b7b2-1bcc92208922-ca-trust-extracted\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkqx8\" (UniqueName: \"kubernetes.io/projected/a4e78f7d-9f01-4fa3-8913-246be64bd119-kube-api-access-rkqx8\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.165918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-registry-certificates\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-installation-pull-secrets\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.165918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-bound-sa-token\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.166083 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.166083 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.165942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4e78f7d-9f01-4fa3-8913-246be64bd119-tmp-dir\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.267013 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.266937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrgsj\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-kube-api-access-lrgsj\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267013 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.266978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/707db365-5d14-4f25-b7b2-1bcc92208922-ca-trust-extracted\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkqx8\" (UniqueName: \"kubernetes.io/projected/a4e78f7d-9f01-4fa3-8913-246be64bd119-kube-api-access-rkqx8\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.267230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-registry-certificates\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267327 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5zfp\" (UniqueName: \"kubernetes.io/projected/30e3c265-ecf4-40d5-910e-cde6902ec81e-kube-api-access-w5zfp\") pod \"ingress-canary-rpbq7\" (UID: \"30e3c265-ecf4-40d5-910e-cde6902ec81e\") " pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.267379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267328    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/707db365-5d14-4f25-b7b2-1bcc92208922-ca-trust-extracted\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267417    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-installation-pull-secrets\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267410    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-bound-sa-token\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267458    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267472    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcc8959-5xkt9: secret "image-registry-tls" not found
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.267538 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267538    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls podName:707db365-5d14-4f25-b7b2-1bcc92208922 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:53.767517508 +0000 UTC m=+34.620844052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls") pod "image-registry-c6bcc8959-5xkt9" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922") : secret "image-registry-tls" not found
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs podName:f320251f-e6c0-482a-b868-2041a175a057 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:25.267547334 +0000 UTC m=+66.120873867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs") pod "network-metrics-daemon-fjkbr" (UID: "f320251f-e6c0-482a-b868-2041a175a057") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267585    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4e78f7d-9f01-4fa3-8913-246be64bd119-tmp-dir\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.267641    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls podName:a4e78f7d-9f01-4fa3-8913-246be64bd119 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:53.767625242 +0000 UTC m=+34.620951762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls") pod "dns-default-pqdb6" (UID: "a4e78f7d-9f01-4fa3-8913-246be64bd119") : secret "dns-default-metrics-tls" not found
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-image-registry-private-configuration\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30e3c265-ecf4-40d5-910e-cde6902ec81e-cert\") pod \"ingress-canary-rpbq7\" (UID: \"30e3c265-ecf4-40d5-910e-cde6902ec81e\") " pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-trusted-ca\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e78f7d-9f01-4fa3-8913-246be64bd119-config-volume\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.267910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267875    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-registry-certificates\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.268292 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.267939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4e78f7d-9f01-4fa3-8913-246be64bd119-tmp-dir\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.268292 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.268212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e78f7d-9f01-4fa3-8913-246be64bd119-config-volume\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.268981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.268844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-trusted-ca\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.271720 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.271699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-image-registry-private-configuration\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.271805 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.271699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-installation-pull-secrets\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.274722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.274694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkqx8\" (UniqueName: \"kubernetes.io/projected/a4e78f7d-9f01-4fa3-8913-246be64bd119-kube-api-access-rkqx8\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.275775 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.275752    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-bound-sa-token\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.276089 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.276060    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrgsj\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-kube-api-access-lrgsj\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.368389 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.368351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5zfp\" (UniqueName: \"kubernetes.io/projected/30e3c265-ecf4-40d5-910e-cde6902ec81e-kube-api-access-w5zfp\") pod \"ingress-canary-rpbq7\" (UID: \"30e3c265-ecf4-40d5-910e-cde6902ec81e\") " pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.368577 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.368416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:53.368577 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.368524    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:53.368577 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.368556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30e3c265-ecf4-40d5-910e-cde6902ec81e-cert\") pod \"ingress-canary-rpbq7\" (UID: \"30e3c265-ecf4-40d5-910e-cde6902ec81e\") " pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.368708 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.368608    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret podName:7e5143b3-55f1-4714-8a04-a2d8f581b307 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:25.368588583 +0000 UTC m=+66.221915104 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret") pod "global-pull-secret-syncer-4mrph" (UID: "7e5143b3-55f1-4714-8a04-a2d8f581b307") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:53.370941 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.370920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30e3c265-ecf4-40d5-910e-cde6902ec81e-cert\") pod \"ingress-canary-rpbq7\" (UID: \"30e3c265-ecf4-40d5-910e-cde6902ec81e\") " pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.375939 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.375920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5zfp\" (UniqueName: \"kubernetes.io/projected/30e3c265-ecf4-40d5-910e-cde6902ec81e-kube-api-access-w5zfp\") pod \"ingress-canary-rpbq7\" (UID: \"30e3c265-ecf4-40d5-910e-cde6902ec81e\") " pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.401155 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.401137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rpbq7"
Jul 24 22:52:53.468903 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.468875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:53.469092 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.468997    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:53.469092 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.469013    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:53.469092 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.469022    2564 projected.go:196] Error preparing data for projected volume kube-api-access-xwfmb for pod openshift-network-diagnostics/network-check-target-jlnvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:53.469092 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.469069    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb podName:3db13fdd-3021-4d6f-9a4b-68d0abba8f78 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:25.469057399 +0000 UTC m=+66.322383917 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwfmb" (UniqueName: "kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb") pod "network-check-target-jlnvw" (UID: "3db13fdd-3021-4d6f-9a4b-68d0abba8f78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:53.582703 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.582676    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rpbq7"]
Jul 24 22:52:53.586695 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:52:53.586669    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e3c265_ecf4_40d5_910e_cde6902ec81e.slice/crio-cd658a34864b42ef0a58488d01879151888a65edfb0e4e36e120d7d9af58dcd4 WatchSource:0}: Error finding container cd658a34864b42ef0a58488d01879151888a65edfb0e4e36e120d7d9af58dcd4: Status 404 returned error can't find the container with id cd658a34864b42ef0a58488d01879151888a65edfb0e4e36e120d7d9af58dcd4
Jul 24 22:52:53.770652 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.770621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:53.771154 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.770663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:53.771154 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.770775    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:52:53.771154 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.770790    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcc8959-5xkt9: secret "image-registry-tls" not found
Jul 24 22:52:53.771154 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.770802    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:53.771154 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.770849    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls podName:707db365-5d14-4f25-b7b2-1bcc92208922 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:54.770835585 +0000 UTC m=+35.624162107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls") pod "image-registry-c6bcc8959-5xkt9" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922") : secret "image-registry-tls" not found
Jul 24 22:52:53.771154 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:53.770866    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls podName:a4e78f7d-9f01-4fa3-8913-246be64bd119 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:54.770860435 +0000 UTC m=+35.624186953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls") pod "dns-default-pqdb6" (UID: "a4e78f7d-9f01-4fa3-8913-246be64bd119") : secret "dns-default-metrics-tls" not found
Jul 24 22:52:53.877286 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.877253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rpbq7" event={"ID":"30e3c265-ecf4-40d5-910e-cde6902ec81e","Type":"ContainerStarted","Data":"cd658a34864b42ef0a58488d01879151888a65edfb0e4e36e120d7d9af58dcd4"}
Jul 24 22:52:53.879820 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.879795    2564 generic.go:356] "Generic (PLEG): container finished" podID="7275e19b-9824-496c-ab8f-1268d2845102" containerID="e507fd2cd3582454a361e5c00f4a39bdf1d8bd0d162e99306f767a498b5cca6c" exitCode=0
Jul 24 22:52:53.879943 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:53.879838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerDied","Data":"e507fd2cd3582454a361e5c00f4a39bdf1d8bd0d162e99306f767a498b5cca6c"}
Jul 24 22:52:54.688522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.688488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:52:54.688757 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.688488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:52:54.688757 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.688488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:52:54.691012 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.690990    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 22:52:54.691796 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.691772    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9dffr\""
Jul 24 22:52:54.691796 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.691779    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2njwm\""
Jul 24 22:52:54.691796 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.691790    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 22:52:54.692022 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.691805    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 22:52:54.692022 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.691785    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 22:52:54.779817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.779783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:54.780240 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.779841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:54.780240 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:54.779936    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:52:54.780240 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:54.779952    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcc8959-5xkt9: secret "image-registry-tls" not found
Jul 24 22:52:54.780240 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:54.779971    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:54.780240 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:54.780015    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls podName:a4e78f7d-9f01-4fa3-8913-246be64bd119 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:56.779997639 +0000 UTC m=+37.633324157 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls") pod "dns-default-pqdb6" (UID: "a4e78f7d-9f01-4fa3-8913-246be64bd119") : secret "dns-default-metrics-tls" not found
Jul 24 22:52:54.780240 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:54.780032    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls podName:707db365-5d14-4f25-b7b2-1bcc92208922 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:56.78002329 +0000 UTC m=+37.633349813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls") pod "image-registry-c6bcc8959-5xkt9" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922") : secret "image-registry-tls" not found
Jul 24 22:52:54.885625 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.885590    2564 generic.go:356] "Generic (PLEG): container finished" podID="7275e19b-9824-496c-ab8f-1268d2845102" containerID="9b190557d71e8d66613d5fd6d3fbe681e34e0465f4f82ac46e25409e6a574b18" exitCode=0
Jul 24 22:52:54.885786 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.885662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerDied","Data":"9b190557d71e8d66613d5fd6d3fbe681e34e0465f4f82ac46e25409e6a574b18"}
Jul 24 22:52:54.890964 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.890935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:54.972962 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:54.972716    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:55.029023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:55.028981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:55.892037 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:55.892008    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" event={"ID":"7275e19b-9824-496c-ab8f-1268d2845102","Type":"ContainerStarted","Data":"e7206de7310b0cada712293b54ac646e92eed86123e04f2325f9a895d9c46672"}
Jul 24 22:52:55.913728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:55.913683    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nvc8q" podStartSLOduration=8.386994832 podStartE2EDuration="36.91366757s" totalImagesPullingTime="28.526672738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:55.912653811 +0000 UTC m=+36.765980365" watchObservedRunningTime="2026-07-24 22:52:55.91366757 +0000 UTC m=+36.766994109"
Jul 24 22:52:56.033444 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:56.033355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:56.134364 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:56.134333    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:56.794548 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:56.794519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:52:56.794646 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:56.794569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:52:56.794697 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:56.794679    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:52:56.794729 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:56.794691    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:56.794729 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:56.794701    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcc8959-5xkt9: secret "image-registry-tls" not found
Jul 24 22:52:56.794788 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:56.794754    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls podName:a4e78f7d-9f01-4fa3-8913-246be64bd119 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:00.794739225 +0000 UTC m=+41.648065743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls") pod "dns-default-pqdb6" (UID: "a4e78f7d-9f01-4fa3-8913-246be64bd119") : secret "dns-default-metrics-tls" not found
Jul 24 22:52:56.794788 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:52:56.794769    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls podName:707db365-5d14-4f25-b7b2-1bcc92208922 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:00.79476311 +0000 UTC m=+41.648089628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls") pod "image-registry-c6bcc8959-5xkt9" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922") : secret "image-registry-tls" not found
Jul 24 22:52:56.895486 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:56.895451    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rpbq7" event={"ID":"30e3c265-ecf4-40d5-910e-cde6902ec81e","Type":"ContainerStarted","Data":"2fb373db2353a600d47c0acd72f9c7edcef7189b8379a82f052db56bbfb85eae"}
Jul 24 22:52:56.917679 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:56.917625    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rpbq7" podStartSLOduration=1.093704841 podStartE2EDuration="3.917611393s" totalImagesPullingTime="2.823906552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:56.916515783 +0000 UTC m=+37.769842338" watchObservedRunningTime="2026-07-24 22:52:56.917611393 +0000 UTC m=+37.770937933"
Jul 24 22:52:58.685706 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:58.685669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:59.725893 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:59.725864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:59.781594 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:59.781567    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:59.921423 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:59.921397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:52:59.986807 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:52:59.986721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:00.823288 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:00.823243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:00.823684 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:00.823296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:00.823684 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:00.823390    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:53:00.823684 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:00.823404    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:00.823684 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:00.823428    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcc8959-5xkt9: secret "image-registry-tls" not found
Jul 24 22:53:00.823684 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:00.823441    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls podName:a4e78f7d-9f01-4fa3-8913-246be64bd119 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:08.82342772 +0000 UTC m=+49.676754238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls") pod "dns-default-pqdb6" (UID: "a4e78f7d-9f01-4fa3-8913-246be64bd119") : secret "dns-default-metrics-tls" not found
Jul 24 22:53:00.823684 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:00.823474    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls podName:707db365-5d14-4f25-b7b2-1bcc92208922 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:08.823459721 +0000 UTC m=+49.676786243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls") pod "image-registry-c6bcc8959-5xkt9" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922") : secret "image-registry-tls" not found
Jul 24 22:53:01.489916 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.489878    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"]
Jul 24 22:53:01.520791 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.520760    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-8qwwz"]
Jul 24 22:53:01.520964 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.520885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.523489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.523453    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 22:53:01.523802 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.523786    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 22:53:01.525649 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.525500    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 22:53:01.525649 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.525600    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gpkzx\""
Jul 24 22:53:01.526502 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.526483    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 22:53:01.534503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.534482    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-8qwwz"]
Jul 24 22:53:01.534581 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.534512    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"]
Jul 24 22:53:01.534581 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.534527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.536780 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.536759    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 24 22:53:01.536949 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.536933    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 24 22:53:01.537061 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.537044    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:53:01.537107 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.537059    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kfwbp\""
Jul 24 22:53:01.537107 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.537050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 24 22:53:01.542916 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.542898    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 24 22:53:01.580025 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.579996    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"]
Jul 24 22:53:01.599500 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.599469    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"]
Jul 24 22:53:01.599631 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.599594    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"
Jul 24 22:53:01.605558 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.605536    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-d7hbx\""
Jul 24 22:53:01.617837 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.617815    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"]
Jul 24 22:53:01.617978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.617948    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:01.620258 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.620241    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 24 22:53:01.620438 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.620421    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 24 22:53:01.620509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.620464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2swtz\""
Jul 24 22:53:01.627953 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.627933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-serving-cert\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.628040 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.627959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb28c\" (UniqueName: \"kubernetes.io/projected/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-kube-api-access-mb28c\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.628040 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.627984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-trusted-ca\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.628040 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.628010    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-config\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.628155 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.628077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.628155 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.628115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxpvx\" (UniqueName: \"kubernetes.io/projected/f7bcc154-1403-44bb-99b5-b0d2c7d23168-kube-api-access-fxpvx\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.628155 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.628149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f7bcc154-1403-44bb-99b5-b0d2c7d23168-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.637894 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.637873    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"]
Jul 24 22:53:01.637994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.637903    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"]
Jul 24 22:53:01.637994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.637916    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"]
Jul 24 22:53:01.637994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.637985    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.640590 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.640572    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 24 22:53:01.640696 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.640610    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:53:01.640763 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.640745    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2x5hq\""
Jul 24 22:53:01.640763 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.640757    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 24 22:53:01.640870 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.640838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 24 22:53:01.728771 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-serving-cert\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.728978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb28c\" (UniqueName: \"kubernetes.io/projected/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-kube-api-access-mb28c\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.728978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:01.728978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-trusted-ca\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.728978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728892    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-config\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.728978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.728978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.728972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd52e67e-d081-4e6f-93b8-5579f43a4537-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:01.729312 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.729004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxpvx\" (UniqueName: \"kubernetes.io/projected/f7bcc154-1403-44bb-99b5-b0d2c7d23168-kube-api-access-fxpvx\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.729312 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.729033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f7bcc154-1403-44bb-99b5-b0d2c7d23168-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.729312 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:01.729065    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:01.729312 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.729141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6crnx\" (UniqueName: \"kubernetes.io/projected/89960639-403b-48c9-beba-1a2e2457ce8b-kube-api-access-6crnx\") pod \"network-check-source-7d88cb55d8-6g78k\" (UID: \"89960639-403b-48c9-beba-1a2e2457ce8b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"
Jul 24 22:53:01.729312 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:01.729163    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls podName:f7bcc154-1403-44bb-99b5-b0d2c7d23168 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:02.229143582 +0000 UTC m=+43.082470123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-h2xp6" (UID: "f7bcc154-1403-44bb-99b5-b0d2c7d23168") : secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:01.729676 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.729653    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-config\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.729804 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.729785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f7bcc154-1403-44bb-99b5-b0d2c7d23168-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.729867 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.729850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-trusted-ca\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.732889 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.732873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-serving-cert\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.737980 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.737958    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb28c\" (UniqueName: \"kubernetes.io/projected/232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3-kube-api-access-mb28c\") pod \"console-operator-584dcb6b54-8qwwz\" (UID: \"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3\") " pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.738067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.738003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxpvx\" (UniqueName: \"kubernetes.io/projected/f7bcc154-1403-44bb-99b5-b0d2c7d23168-kube-api-access-fxpvx\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:01.769271 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.769195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:01.829918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.829878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf7bl\" (UniqueName: \"kubernetes.io/projected/94ea935e-eeaf-4f1c-9acb-8e108377860a-kube-api-access-gf7bl\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.829938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94ea935e-eeaf-4f1c-9acb-8e108377860a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.830010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd52e67e-d081-4e6f-93b8-5579f43a4537-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.830071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.830107    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94ea935e-eeaf-4f1c-9acb-8e108377860a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.830171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6crnx\" (UniqueName: \"kubernetes.io/projected/89960639-403b-48c9-beba-1a2e2457ce8b-kube-api-access-6crnx\") pod \"network-check-source-7d88cb55d8-6g78k\" (UID: \"89960639-403b-48c9-beba-1a2e2457ce8b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:01.830210    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 22:53:01.830387 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:01.830287    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert podName:bd52e67e-d081-4e6f-93b8-5579f43a4537 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:02.330273367 +0000 UTC m=+43.183599885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4fl6d" (UID: "bd52e67e-d081-4e6f-93b8-5579f43a4537") : secret "networking-console-plugin-cert" not found
Jul 24 22:53:01.830838 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.830818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd52e67e-d081-4e6f-93b8-5579f43a4537-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:01.838954 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.838938    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6crnx\" (UniqueName: \"kubernetes.io/projected/89960639-403b-48c9-beba-1a2e2457ce8b-kube-api-access-6crnx\") pod \"network-check-source-7d88cb55d8-6g78k\" (UID: \"89960639-403b-48c9-beba-1a2e2457ce8b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"
Jul 24 22:53:01.843736 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.843717    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:01.869321 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.869299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:01.910317 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.910278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"
Jul 24 22:53:01.927878 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.927850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:01.931479 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.931443    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94ea935e-eeaf-4f1c-9acb-8e108377860a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.931598 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.931582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf7bl\" (UniqueName: \"kubernetes.io/projected/94ea935e-eeaf-4f1c-9acb-8e108377860a-kube-api-access-gf7bl\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.931663 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.931629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94ea935e-eeaf-4f1c-9acb-8e108377860a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.931985 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.931964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94ea935e-eeaf-4f1c-9acb-8e108377860a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.936230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.936182    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94ea935e-eeaf-4f1c-9acb-8e108377860a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.939479 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.939427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf7bl\" (UniqueName: \"kubernetes.io/projected/94ea935e-eeaf-4f1c-9acb-8e108377860a-kube-api-access-gf7bl\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88\" (UID: \"94ea935e-eeaf-4f1c-9acb-8e108377860a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:01.946416 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:01.946396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"
Jul 24 22:53:02.022473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.022388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-8qwwz"]
Jul 24 22:53:02.025774 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:02.025714    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232174b4_8fff_4aeb_8d4b_13a5dd3f4bb3.slice/crio-921f90b572345c4f8d40a9ffda012061fc272ed6693103fe086c94514ba25cce WatchSource:0}: Error finding container 921f90b572345c4f8d40a9ffda012061fc272ed6693103fe086c94514ba25cce: Status 404 returned error can't find the container with id 921f90b572345c4f8d40a9ffda012061fc272ed6693103fe086c94514ba25cce
Jul 24 22:53:02.057649 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.057626    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k"]
Jul 24 22:53:02.060834 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:02.060809    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89960639_403b_48c9_beba_1a2e2457ce8b.slice/crio-415f7c84166876d566ef6d0f818e8172ffbbe0930fe3f372d7990ccee3c654d3 WatchSource:0}: Error finding container 415f7c84166876d566ef6d0f818e8172ffbbe0930fe3f372d7990ccee3c654d3: Status 404 returned error can't find the container with id 415f7c84166876d566ef6d0f818e8172ffbbe0930fe3f372d7990ccee3c654d3
Jul 24 22:53:02.103067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.103039    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88"]
Jul 24 22:53:02.105715 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:02.105693    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ea935e_eeaf_4f1c_9acb_8e108377860a.slice/crio-4c20f7e15170795dac8c9036ac7ca12ed2a9b53fa95cb76572ef4e2e195fd60b WatchSource:0}: Error finding container 4c20f7e15170795dac8c9036ac7ca12ed2a9b53fa95cb76572ef4e2e195fd60b: Status 404 returned error can't find the container with id 4c20f7e15170795dac8c9036ac7ca12ed2a9b53fa95cb76572ef4e2e195fd60b
Jul 24 22:53:02.233993 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.233959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:02.234162 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:02.234107    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:02.234209 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:02.234198    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls podName:f7bcc154-1403-44bb-99b5-b0d2c7d23168 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:03.23418014 +0000 UTC m=+44.087506658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-h2xp6" (UID: "f7bcc154-1403-44bb-99b5-b0d2c7d23168") : secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:02.334949 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.334926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:02.335088 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:02.335041    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 22:53:02.335144 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:02.335092    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert podName:bd52e67e-d081-4e6f-93b8-5579f43a4537 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:03.335078438 +0000 UTC m=+44.188404956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4fl6d" (UID: "bd52e67e-d081-4e6f-93b8-5579f43a4537") : secret "networking-console-plugin-cert" not found
Jul 24 22:53:02.913408 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.913369    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k" event={"ID":"89960639-403b-48c9-beba-1a2e2457ce8b","Type":"ContainerStarted","Data":"415f7c84166876d566ef6d0f818e8172ffbbe0930fe3f372d7990ccee3c654d3"}
Jul 24 22:53:02.915181 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.915148    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88" event={"ID":"94ea935e-eeaf-4f1c-9acb-8e108377860a","Type":"ContainerStarted","Data":"4c20f7e15170795dac8c9036ac7ca12ed2a9b53fa95cb76572ef4e2e195fd60b"}
Jul 24 22:53:02.916626 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:02.916592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" event={"ID":"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3","Type":"ContainerStarted","Data":"921f90b572345c4f8d40a9ffda012061fc272ed6693103fe086c94514ba25cce"}
Jul 24 22:53:03.244219 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:03.244134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:03.244382 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:03.244359    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:03.244439 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:03.244424    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls podName:f7bcc154-1403-44bb-99b5-b0d2c7d23168 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:05.244404984 +0000 UTC m=+46.097731512 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-h2xp6" (UID: "f7bcc154-1403-44bb-99b5-b0d2c7d23168") : secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:03.271903 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:03.271839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:03.345368 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:03.345333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:03.345618 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:03.345552    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 22:53:03.345736 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:03.345628    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert podName:bd52e67e-d081-4e6f-93b8-5579f43a4537 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:05.345606933 +0000 UTC m=+46.198933453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4fl6d" (UID: "bd52e67e-d081-4e6f-93b8-5579f43a4537") : secret "networking-console-plugin-cert" not found
Jul 24 22:53:03.372730 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:03.372698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:04.684939 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:04.684905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:04.737384 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:04.737353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:05.261093 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:05.261050    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:05.261327 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:05.261220    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:05.261327 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:05.261298    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls podName:f7bcc154-1403-44bb-99b5-b0d2c7d23168 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:09.261280734 +0000 UTC m=+50.114607257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-h2xp6" (UID: "f7bcc154-1403-44bb-99b5-b0d2c7d23168") : secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:05.361846 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:05.361807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:05.362031 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:05.361938    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 22:53:05.362031 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:05.362017    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert podName:bd52e67e-d081-4e6f-93b8-5579f43a4537 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:09.361997561 +0000 UTC m=+50.215324102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4fl6d" (UID: "bd52e67e-d081-4e6f-93b8-5579f43a4537") : secret "networking-console-plugin-cert" not found
Jul 24 22:53:06.246360 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.246340    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fps4r"]
Jul 24 22:53:06.267951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.267919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:06.269243 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.269219    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fps4r"]
Jul 24 22:53:06.269349 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.269331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.271681 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.271663    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 22:53:06.271783 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.271716    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 22:53:06.271783 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.271761    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 22:53:06.271881 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.271851    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 22:53:06.272962 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.272941    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mzp5b\""
Jul 24 22:53:06.325368 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.325335    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:06.369578 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.369556    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.369662 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.369601    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ad14177-7c17-4970-97e9-f22e67ee5712-crio-socket\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.369662 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.369631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.369752 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.369685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ad14177-7c17-4970-97e9-f22e67ee5712-data-volume\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.369752 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.369731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlfxn\" (UniqueName: \"kubernetes.io/projected/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-api-access-wlfxn\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470039 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlfxn\" (UniqueName: \"kubernetes.io/projected/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-api-access-wlfxn\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470114 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470241 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ad14177-7c17-4970-97e9-f22e67ee5712-crio-socket\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470241 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470241 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.470226    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:06.470398 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.470282    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:06.970263267 +0000 UTC m=+47.823589785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:06.470398 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.470310    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 22:53:06.470398 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ad14177-7c17-4970-97e9-f22e67ee5712-data-volume\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470398 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.470360    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:06.970344438 +0000 UTC m=+47.823670957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : configmap "kube-rbac-proxy" not found
Jul 24 22:53:06.470398 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ad14177-7c17-4970-97e9-f22e67ee5712-crio-socket\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.470645 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.470556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ad14177-7c17-4970-97e9-f22e67ee5712-data-volume\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.479915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.479893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlfxn\" (UniqueName: \"kubernetes.io/projected/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-api-access-wlfxn\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.884095 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.884068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:06.927733 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.927700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88" event={"ID":"94ea935e-eeaf-4f1c-9acb-8e108377860a","Type":"ContainerStarted","Data":"c51430bca81ec04c347c2269143c0f0063c45f2a9f13ebfb1b13734e3d8b22cc"}
Jul 24 22:53:06.929336 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.929318    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/0.log"
Jul 24 22:53:06.929461 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.929354    2564 generic.go:356] "Generic (PLEG): container finished" podID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3" containerID="bdb8ffa842cb39f6ec89d2efcbcbd4f843f7bade54566a9b4a81af5ce8e00239" exitCode=255
Jul 24 22:53:06.929461 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.929381    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" event={"ID":"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3","Type":"ContainerDied","Data":"bdb8ffa842cb39f6ec89d2efcbcbd4f843f7bade54566a9b4a81af5ce8e00239"}
Jul 24 22:53:06.929593 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.929580    2564 scope.go:117] "RemoveContainer" containerID="bdb8ffa842cb39f6ec89d2efcbcbd4f843f7bade54566a9b4a81af5ce8e00239"
Jul 24 22:53:06.930939 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.930920    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k" event={"ID":"89960639-403b-48c9-beba-1a2e2457ce8b","Type":"ContainerStarted","Data":"7151b3378960f647908d16a6e36e0da2407e14b0287a7ba1cdff460e363527d2"}
Jul 24 22:53:06.942946 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.942904    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88" podStartSLOduration=1.581259787 podStartE2EDuration="5.94288629s" totalImagesPullingTime="4.361626503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:06.942375106 +0000 UTC m=+47.795701647" watchObservedRunningTime="2026-07-24 22:53:06.94288629 +0000 UTC m=+47.796212827"
Jul 24 22:53:06.943529 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.943507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:06.956492 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.956453    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6g78k" podStartSLOduration=1.5402408269999999 podStartE2EDuration="5.956438839s" totalImagesPullingTime="4.416198012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:06.955536961 +0000 UTC m=+47.808863504" watchObservedRunningTime="2026-07-24 22:53:06.956438839 +0000 UTC m=+47.809765380"
Jul 24 22:53:06.974331 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.974308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.974441 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:06.974421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:06.974539 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.974458    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:06.974539 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.974521    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:07.974501144 +0000 UTC m=+48.827827664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:06.974991 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.974835    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 22:53:06.974991 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:06.974886    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:07.974869912 +0000 UTC m=+48.828196433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : configmap "kube-rbac-proxy" not found
Jul 24 22:53:07.308978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.308889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:07.408797 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.408749    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:07.508530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.508494    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:07.609468 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.609439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:07.709493 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.709465    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:07.934403 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.934337    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/1.log"
Jul 24 22:53:07.934746 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.934730    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/0.log"
Jul 24 22:53:07.934839 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.934769    2564 generic.go:356] "Generic (PLEG): container finished" podID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3" containerID="34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4" exitCode=255
Jul 24 22:53:07.934899 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.934853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" event={"ID":"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3","Type":"ContainerDied","Data":"34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4"}
Jul 24 22:53:07.934940 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.934901    2564 scope.go:117] "RemoveContainer" containerID="bdb8ffa842cb39f6ec89d2efcbcbd4f843f7bade54566a9b4a81af5ce8e00239"
Jul 24 22:53:07.935094 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.935060    2564 scope.go:117] "RemoveContainer" containerID="34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4"
Jul 24 22:53:07.935281 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:07.935264    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8qwwz_openshift-console-operator(232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" podUID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3"
Jul 24 22:53:07.984137 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.984097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:07.984294 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:07.984279    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 22:53:07.984625 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:07.984412    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:09.984388679 +0000 UTC m=+50.837715200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : configmap "kube-rbac-proxy" not found
Jul 24 22:53:07.984625 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:07.984479    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:07.984625 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:07.984551    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:07.984625 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:07.984593    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:09.984581069 +0000 UTC m=+50.837907588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:08.035592 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.035573    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9vcd8_02648613-32e6-4cc7-aaab-e39b8876124d/dns-node-resolver/0.log"
Jul 24 22:53:08.505174 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.505138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:08.604897 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.604859    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:08.890623 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.890596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:08.890757 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.890662    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:08.890798 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:08.890758    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:08.890798 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:08.890773    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcc8959-5xkt9: secret "image-registry-tls" not found
Jul 24 22:53:08.890859 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:08.890810    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:53:08.890859 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:08.890828    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls podName:707db365-5d14-4f25-b7b2-1bcc92208922 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:24.890811725 +0000 UTC m=+65.744138252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls") pod "image-registry-c6bcc8959-5xkt9" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922") : secret "image-registry-tls" not found
Jul 24 22:53:08.890859 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:08.890858    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls podName:a4e78f7d-9f01-4fa3-8913-246be64bd119 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:24.890844305 +0000 UTC m=+65.744170822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls") pod "dns-default-pqdb6" (UID: "a4e78f7d-9f01-4fa3-8913-246be64bd119") : secret "dns-default-metrics-tls" not found
Jul 24 22:53:08.938376 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.938354    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/1.log"
Jul 24 22:53:08.938643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:08.938629    2564 scope.go:117] "RemoveContainer" containerID="34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4"
Jul 24 22:53:08.938777 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:08.938762    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8qwwz_openshift-console-operator(232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" podUID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3"
Jul 24 22:53:09.228621 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.228526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:09.236682 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.236660    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cn5mp_011616e0-ac21-4ad2-979f-baaed2fd1174/node-ca/0.log"
Jul 24 22:53:09.279867 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.279834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:09.294130 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.294106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:09.294258 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:09.294244    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:09.294316 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:09.294307    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls podName:f7bcc154-1403-44bb-99b5-b0d2c7d23168 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:17.294291755 +0000 UTC m=+58.147618277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-h2xp6" (UID: "f7bcc154-1403-44bb-99b5-b0d2c7d23168") : secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:09.395222 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.395190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:09.395363 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:09.395327    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 22:53:09.395402 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:09.395389    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert podName:bd52e67e-d081-4e6f-93b8-5579f43a4537 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:17.395375728 +0000 UTC m=+58.248702249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4fl6d" (UID: "bd52e67e-d081-4e6f-93b8-5579f43a4537") : secret "networking-console-plugin-cert" not found
Jul 24 22:53:09.437450 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.437425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:09.855826 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.855797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:09.912813 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.912782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:09.998766 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.998739    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:09.998933 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.998824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:09.998933 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:09.998907    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:09.999049 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:09.998954    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:13.998941135 +0000 UTC m=+54.852267653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:09.999339 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:09.999316    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ad14177-7c17-4970-97e9-f22e67ee5712-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:10.215763 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:10.215684    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:10.234026 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:10.234007    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rpbq7_30e3c265-ecf4-40d5-910e-cde6902ec81e/serve-healthcheck-canary/0.log"
Jul 24 22:53:11.844347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:11.844317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:11.844773 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:11.844402    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:11.844773 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:11.844688    2564 scope.go:117] "RemoveContainer" containerID="34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4"
Jul 24 22:53:11.844844 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:11.844817    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-8qwwz_openshift-console-operator(232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" podUID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3"
Jul 24 22:53:12.650298 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:12.650264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:13.693234 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:13.693195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:13.965383 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:13.965290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:14.030556 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:14.030526    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:14.030686 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:14.030670    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:14.030741 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:14.030732    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls podName:9ad14177-7c17-4970-97e9-f22e67ee5712 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:22.030717544 +0000 UTC m=+62.884044061 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fps4r" (UID: "9ad14177-7c17-4970-97e9-f22e67ee5712") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:14.126336 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:14.126309    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:14.275269 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:14.275210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:14.374406 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:14.374383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:14.474442 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:14.474418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:15.193073 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:15.193042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:15.797292 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:15.797258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:16.000874 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:16.000845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:16.189703 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:16.189673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:16.394063 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:16.394032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:17.356789 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:17.356755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:17.356958 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:17.356870    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:17.356958 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:17.356924    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls podName:f7bcc154-1403-44bb-99b5-b0d2c7d23168 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:33.356910173 +0000 UTC m=+74.210236691 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-h2xp6" (UID: "f7bcc154-1403-44bb-99b5-b0d2c7d23168") : secret "cluster-monitoring-operator-tls" not found
Jul 24 22:53:17.410636 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:17.410600    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:17.458611 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:17.458569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:17.461132 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:17.458812    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 22:53:17.461132 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:17.458921    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert podName:bd52e67e-d081-4e6f-93b8-5579f43a4537 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:33.458897471 +0000 UTC m=+74.312224004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4fl6d" (UID: "bd52e67e-d081-4e6f-93b8-5579f43a4537") : secret "networking-console-plugin-cert" not found
Jul 24 22:53:17.504232 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:17.504193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:17.604378 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:17.604341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:17.994745 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:17.994711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:18.189997 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:18.189962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:18.240084 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:18.240057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:18.796579 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:18.796531    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:18.874273 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:18.874250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qcdfg"
Jul 24 22:53:19.204272 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:19.204241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:19.590790 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:19.590757    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:19.793486 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:19.793443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:20.031650 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:20.031550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:20.392942 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:20.392908    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:20.992929 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:20.992883    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:21.058396 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:21.058366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:21.197475 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:21.197447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:21.392483 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:21.392455    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:21.787960 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:21.787882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:22.096249 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.096222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:22.098347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.098329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ad14177-7c17-4970-97e9-f22e67ee5712-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fps4r\" (UID: \"9ad14177-7c17-4970-97e9-f22e67ee5712\") " pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:22.183935 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.183909    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mzp5b\""
Jul 24 22:53:22.192597 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.192568    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fps4r"
Jul 24 22:53:22.309770 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.309740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fps4r"]
Jul 24 22:53:22.313477 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:22.313449    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad14177_7c17_4970_97e9_f22e67ee5712.slice/crio-6d655c29ec2fbe8dda326231c75df799931861bfdec96cdacda32c538686cb8a WatchSource:0}: Error finding container 6d655c29ec2fbe8dda326231c75df799931861bfdec96cdacda32c538686cb8a: Status 404 returned error can't find the container with id 6d655c29ec2fbe8dda326231c75df799931861bfdec96cdacda32c538686cb8a
Jul 24 22:53:22.589054 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.589025    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:22.689410 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.689383    2564 scope.go:117] "RemoveContainer" containerID="34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4"
Jul 24 22:53:22.790806 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.790772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:22.973352 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.973270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fps4r" event={"ID":"9ad14177-7c17-4970-97e9-f22e67ee5712","Type":"ContainerStarted","Data":"6d655c29ec2fbe8dda326231c75df799931861bfdec96cdacda32c538686cb8a"}
Jul 24 22:53:22.975795 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.975773    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 22:53:22.976265 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.976247    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/1.log"
Jul 24 22:53:22.976372 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.976281    2564 generic.go:356] "Generic (PLEG): container finished" podID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3" containerID="56c42b4ab2a84d0b2295e61ff10c9160f5ab6febdca03bc27787bc89fdf80a10" exitCode=255
Jul 24 22:53:22.976372 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.976308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" event={"ID":"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3","Type":"ContainerDied","Data":"56c42b4ab2a84d0b2295e61ff10c9160f5ab6febdca03bc27787bc89fdf80a10"}
Jul 24 22:53:22.976372 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.976354    2564 scope.go:117] "RemoveContainer" containerID="34072a1cf7aacf7f4fd15b966cec02a88b71af65a8aec799ce2dd38b2e1f8ae4"
Jul 24 22:53:22.976777 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:22.976761    2564 scope.go:117] "RemoveContainer" containerID="56c42b4ab2a84d0b2295e61ff10c9160f5ab6febdca03bc27787bc89fdf80a10"
Jul 24 22:53:22.976962 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:22.976942    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-8qwwz_openshift-console-operator(232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" podUID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3"
Jul 24 22:53:23.194530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:23.194485    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:23.401709 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:23.401661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:23.794226 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:23.794142    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:23.980431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:23.980403    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 22:53:24.192563 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:24.192527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:24.390031 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:24.389997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:24.921371 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:24.921336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:24.921547 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:24.921398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:24.923996 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:24.923967    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4e78f7d-9f01-4fa3-8913-246be64bd119-metrics-tls\") pod \"dns-default-pqdb6\" (UID: \"a4e78f7d-9f01-4fa3-8913-246be64bd119\") " pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:24.924153 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:24.923997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"image-registry-c6bcc8959-5xkt9\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") " pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:25.181352 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.181283    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tmxk\""
Jul 24 22:53:25.189358 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.189334    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:25.196208 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.196184    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j4zlm\""
Jul 24 22:53:25.204840 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.204819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:25.324090 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.324056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:53:25.326634 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.326615    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 22:53:25.337724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.337698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f320251f-e6c0-482a-b868-2041a175a057-metrics-certs\") pod \"network-metrics-daemon-fjkbr\" (UID: \"f320251f-e6c0-482a-b868-2041a175a057\") " pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:53:25.425172 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.425113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:53:25.427650 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.427630    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 22:53:25.438718 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.438658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e5143b3-55f1-4714-8a04-a2d8f581b307-original-pull-secret\") pod \"global-pull-secret-syncer-4mrph\" (UID: \"7e5143b3-55f1-4714-8a04-a2d8f581b307\") " pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:53:25.526573 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.526306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:53:25.529344 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.529312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwfmb\" (UniqueName: \"kubernetes.io/projected/3db13fdd-3021-4d6f-9a4b-68d0abba8f78-kube-api-access-xwfmb\") pod \"network-check-target-jlnvw\" (UID: \"3db13fdd-3021-4d6f-9a4b-68d0abba8f78\") " pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:53:25.601016 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.600979    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9dffr\""
Jul 24 22:53:25.604858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.604834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mrph"
Jul 24 22:53:25.608983 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.608951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:53:25.612383 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.612358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2njwm\""
Jul 24 22:53:25.621339 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:25.621319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjkbr"
Jul 24 22:53:27.333403 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.332940    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"]
Jul 24 22:53:27.335979 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.335951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.338997 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.338738    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 22:53:27.339798 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.339766    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 22:53:27.340005 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.339988    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-c5865\""
Jul 24 22:53:27.340418 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.340220    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 22:53:27.340418 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.340312    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 22:53:27.355893 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.355871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"]
Jul 24 22:53:27.356994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.356974    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jlnvw"]
Jul 24 22:53:27.365727 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:27.365702    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db13fdd_3021_4d6f_9a4b_68d0abba8f78.slice/crio-cf18551602ed8b66178c57a7274bcc3e14cc623dd48555b98a32f55549e10eb6 WatchSource:0}: Error finding container cf18551602ed8b66178c57a7274bcc3e14cc623dd48555b98a32f55549e10eb6: Status 404 returned error can't find the container with id cf18551602ed8b66178c57a7274bcc3e14cc623dd48555b98a32f55549e10eb6
Jul 24 22:53:27.443475 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.443412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f650190c-6be3-4909-adc6-dea29f262ad6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56478cbffd-6d22d\" (UID: \"f650190c-6be3-4909-adc6-dea29f262ad6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.443589 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.443549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54xtq\" (UniqueName: \"kubernetes.io/projected/f650190c-6be3-4909-adc6-dea29f262ad6-kube-api-access-54xtq\") pod \"managed-serviceaccount-addon-agent-56478cbffd-6d22d\" (UID: \"f650190c-6be3-4909-adc6-dea29f262ad6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.476300 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.476270    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:27.544225 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.544192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f650190c-6be3-4909-adc6-dea29f262ad6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56478cbffd-6d22d\" (UID: \"f650190c-6be3-4909-adc6-dea29f262ad6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.544413 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.544260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54xtq\" (UniqueName: \"kubernetes.io/projected/f650190c-6be3-4909-adc6-dea29f262ad6-kube-api-access-54xtq\") pod \"managed-serviceaccount-addon-agent-56478cbffd-6d22d\" (UID: \"f650190c-6be3-4909-adc6-dea29f262ad6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.547011 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.546982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f650190c-6be3-4909-adc6-dea29f262ad6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56478cbffd-6d22d\" (UID: \"f650190c-6be3-4909-adc6-dea29f262ad6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.555036 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.555007    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54xtq\" (UniqueName: \"kubernetes.io/projected/f650190c-6be3-4909-adc6-dea29f262ad6-kube-api-access-54xtq\") pod \"managed-serviceaccount-addon-agent-56478cbffd-6d22d\" (UID: \"f650190c-6be3-4909-adc6-dea29f262ad6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.569951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.569919    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pqdb6"]
Jul 24 22:53:27.572263 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:27.572233    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e78f7d_9f01_4fa3_8913_246be64bd119.slice/crio-772d325c4599d8c4104501bdb44e47a5bd4a5420cc45dff87a4a4b28df7c314f WatchSource:0}: Error finding container 772d325c4599d8c4104501bdb44e47a5bd4a5420cc45dff87a4a4b28df7c314f: Status 404 returned error can't find the container with id 772d325c4599d8c4104501bdb44e47a5bd4a5420cc45dff87a4a4b28df7c314f
Jul 24 22:53:27.573972 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.573918    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjkbr"]
Jul 24 22:53:27.580545 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.577272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6bcc8959-5xkt9"]
Jul 24 22:53:27.580545 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.578997    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4mrph"]
Jul 24 22:53:27.584301 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.583835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:27.660722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.660693    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:27.666562 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.666543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"
Jul 24 22:53:27.814146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.814027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:27.859567 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.859532    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d"]
Jul 24 22:53:27.863212 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:27.863180    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf650190c_6be3_4909_adc6_dea29f262ad6.slice/crio-160098d9c9fb6391c1c63bff199d3b56b8b1063298b863e285e9ba75cbf4d00e WatchSource:0}: Error finding container 160098d9c9fb6391c1c63bff199d3b56b8b1063298b863e285e9ba75cbf4d00e: Status 404 returned error can't find the container with id 160098d9c9fb6391c1c63bff199d3b56b8b1063298b863e285e9ba75cbf4d00e
Jul 24 22:53:27.873791 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.873767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:27.994643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.994566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" event={"ID":"707db365-5d14-4f25-b7b2-1bcc92208922","Type":"ContainerStarted","Data":"0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4"}
Jul 24 22:53:27.994643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.994629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" event={"ID":"707db365-5d14-4f25-b7b2-1bcc92208922","Type":"ContainerStarted","Data":"033896cc68e4789b90fd75c30a636e1bb864d11e52ac1ac88f0a812467a4943a"}
Jul 24 22:53:27.995049 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.995018    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:27.996459 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.996433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4mrph" event={"ID":"7e5143b3-55f1-4714-8a04-a2d8f581b307","Type":"ContainerStarted","Data":"d1a6faf5764267a9fd122244f934b13f740cbf885166497a2befc4826af4ce32"}
Jul 24 22:53:28.000028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:27.999998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fps4r" event={"ID":"9ad14177-7c17-4970-97e9-f22e67ee5712","Type":"ContainerStarted","Data":"1e86a170dbc5f868884aba61380667916b75d529af565681092294be89d7e4e2"}
Jul 24 22:53:28.002503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.002482    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d" event={"ID":"f650190c-6be3-4909-adc6-dea29f262ad6","Type":"ContainerStarted","Data":"160098d9c9fb6391c1c63bff199d3b56b8b1063298b863e285e9ba75cbf4d00e"}
Jul 24 22:53:28.005381 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.005355    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jlnvw" event={"ID":"3db13fdd-3021-4d6f-9a4b-68d0abba8f78","Type":"ContainerStarted","Data":"c724863caefd862a207c5b692f649300fe950a8339f9b3fdf2a94b688cf47a43"}
Jul 24 22:53:28.005473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.005398    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jlnvw" event={"ID":"3db13fdd-3021-4d6f-9a4b-68d0abba8f78","Type":"ContainerStarted","Data":"cf18551602ed8b66178c57a7274bcc3e14cc623dd48555b98a32f55549e10eb6"}
Jul 24 22:53:28.005913 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.005851    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:53:28.010114 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.010057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pqdb6" event={"ID":"a4e78f7d-9f01-4fa3-8913-246be64bd119","Type":"ContainerStarted","Data":"772d325c4599d8c4104501bdb44e47a5bd4a5420cc45dff87a4a4b28df7c314f"}
Jul 24 22:53:28.012953 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.012869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjkbr" event={"ID":"f320251f-e6c0-482a-b868-2041a175a057","Type":"ContainerStarted","Data":"222ac64040bbd9688ca89340c45bed5771468f5608559a9847c8a14afb7962b5"}
Jul 24 22:53:28.014841 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:28.014451    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" podStartSLOduration=68.014433517 podStartE2EDuration="1m8.014433517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:28.013209924 +0000 UTC m=+68.866536466" watchObservedRunningTime="2026-07-24 22:53:28.014433517 +0000 UTC m=+68.867760058"
Jul 24 22:53:29.021208 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:29.021174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjkbr" event={"ID":"f320251f-e6c0-482a-b868-2041a175a057","Type":"ContainerStarted","Data":"c22b9fbb2e7eeec79c1eec1aec351404eab6e00ea32ba65d9b0a115f5ef05f67"}
Jul 24 22:53:29.821165 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:29.821049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:29.921673 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:29.921632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:30.031698 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:30.031624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjkbr" event={"ID":"f320251f-e6c0-482a-b868-2041a175a057","Type":"ContainerStarted","Data":"ddce33cf8c60c0ebae19f740ee5eadf9f13a80687c6083e6bbbc6510f8ae911c"}
Jul 24 22:53:30.051589 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:30.051530    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jlnvw" podStartSLOduration=70.051508972 podStartE2EDuration="1m10.051508972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:28.028013883 +0000 UTC m=+68.881340425" watchObservedRunningTime="2026-07-24 22:53:30.051508972 +0000 UTC m=+70.904835513"
Jul 24 22:53:30.051755 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:30.051685    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fjkbr" podStartSLOduration=69.79079949 podStartE2EDuration="1m11.051677224s" totalImagesPullingTime="1.260877734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:30.050798431 +0000 UTC m=+70.904124974" watchObservedRunningTime="2026-07-24 22:53:30.051677224 +0000 UTC m=+70.905003765"
Jul 24 22:53:30.146676 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:30.146641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:30.244782 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:30.244744    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:30.307079 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:30.307034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:31.844039 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:31.844004    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:31.844469 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:31.844078    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:31.844545 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:31.844485    2564 scope.go:117] "RemoveContainer" containerID="56c42b4ab2a84d0b2295e61ff10c9160f5ab6febdca03bc27787bc89fdf80a10"
Jul 24 22:53:31.844693 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:31.844667    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-8qwwz_openshift-console-operator(232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" podUID="232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3"
Jul 24 22:53:33.394488 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.394452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:33.396724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.396693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f7bcc154-1403-44bb-99b5-b0d2c7d23168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-h2xp6\" (UID: \"f7bcc154-1403-44bb-99b5-b0d2c7d23168\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:33.495712 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.495672    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:33.498452 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.498425    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd52e67e-d081-4e6f-93b8-5579f43a4537-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4fl6d\" (UID: \"bd52e67e-d081-4e6f-93b8-5579f43a4537\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:33.633671 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.633640    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gpkzx\""
Jul 24 22:53:33.641932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.641897    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"
Jul 24 22:53:33.728438 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.728410    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2swtz\""
Jul 24 22:53:33.737215 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:33.737179    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"
Jul 24 22:53:34.627238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:34.627147    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d"]
Jul 24 22:53:34.632062 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:34.632033    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd52e67e_d081_4e6f_93b8_5579f43a4537.slice/crio-683cd2738eeb2d55babf21ebacfd3d04323a50142694cb5a00aa433e8c50835c WatchSource:0}: Error finding container 683cd2738eeb2d55babf21ebacfd3d04323a50142694cb5a00aa433e8c50835c: Status 404 returned error can't find the container with id 683cd2738eeb2d55babf21ebacfd3d04323a50142694cb5a00aa433e8c50835c
Jul 24 22:53:34.649668 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:34.649634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6"]
Jul 24 22:53:34.654624 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:34.654593    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bcc154_1403_44bb_99b5_b0d2c7d23168.slice/crio-82de141e72fa95c9130895464d7a80ce2f157d948f1abf90a8186836449c3136 WatchSource:0}: Error finding container 82de141e72fa95c9130895464d7a80ce2f157d948f1abf90a8186836449c3136: Status 404 returned error can't find the container with id 82de141e72fa95c9130895464d7a80ce2f157d948f1abf90a8186836449c3136
Jul 24 22:53:35.048757 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.048672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4mrph" event={"ID":"7e5143b3-55f1-4714-8a04-a2d8f581b307","Type":"ContainerStarted","Data":"4123296b4c8f63de7d7e97563577e55d8af508c61cb02e2c3a8ab946423d4c0a"}
Jul 24 22:53:35.050661 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.050630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fps4r" event={"ID":"9ad14177-7c17-4970-97e9-f22e67ee5712","Type":"ContainerStarted","Data":"67ddde738fb39db6c8229875c1486f05d652c35c153f454958ccdb3be387fcb9"}
Jul 24 22:53:35.052263 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.052238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d" event={"ID":"f650190c-6be3-4909-adc6-dea29f262ad6","Type":"ContainerStarted","Data":"f2a73a743f5ca296a10bb425bdd0d7361144341f2f6051bd4917dca10772fbe0"}
Jul 24 22:53:35.053411 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.053388    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6" event={"ID":"f7bcc154-1403-44bb-99b5-b0d2c7d23168","Type":"ContainerStarted","Data":"82de141e72fa95c9130895464d7a80ce2f157d948f1abf90a8186836449c3136"}
Jul 24 22:53:35.054589 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.054567    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d" event={"ID":"bd52e67e-d081-4e6f-93b8-5579f43a4537","Type":"ContainerStarted","Data":"683cd2738eeb2d55babf21ebacfd3d04323a50142694cb5a00aa433e8c50835c"}
Jul 24 22:53:35.056473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.056450    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pqdb6" event={"ID":"a4e78f7d-9f01-4fa3-8913-246be64bd119","Type":"ContainerStarted","Data":"112dd7ae13801d230d5c124142878ed2f414f4a62dd0a465708e73290ef67c8a"}
Jul 24 22:53:35.056575 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.056493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pqdb6" event={"ID":"a4e78f7d-9f01-4fa3-8913-246be64bd119","Type":"ContainerStarted","Data":"e4612b7a0b5e9a2a299f1f34eb4fba55343ab1c6c3bb04179739644ef52211de"}
Jul 24 22:53:35.056575 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.056521    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:35.063041 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.063004    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4mrph" podStartSLOduration=68.196833074 podStartE2EDuration="1m15.062990779s" totalImagesPullingTime="6.866157705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:35.062255283 +0000 UTC m=+75.915581826" watchObservedRunningTime="2026-07-24 22:53:35.062990779 +0000 UTC m=+75.916317319"
Jul 24 22:53:35.076456 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.076416    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56478cbffd-6d22d" podStartSLOduration=1.4821041240000001 podStartE2EDuration="8.076403373s" totalImagesPullingTime="6.594299249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:35.076113935 +0000 UTC m=+75.929440477" watchObservedRunningTime="2026-07-24 22:53:35.076403373 +0000 UTC m=+75.929729942"
Jul 24 22:53:35.091868 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:35.091820    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pqdb6" podStartSLOduration=35.217431737 podStartE2EDuration="42.091801624s" totalImagesPullingTime="6.874369887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:35.090953893 +0000 UTC m=+75.944280434" watchObservedRunningTime="2026-07-24 22:53:35.091801624 +0000 UTC m=+75.945128191"
Jul 24 22:53:38.880509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:38.880473    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.038628 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.038596    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"]
Jul 24 22:53:39.042372 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.042341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.043047 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.043032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:39.045327 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.045307    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ddlwt\""
Jul 24 22:53:39.045566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.045548    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 24 22:53:39.048756 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.048732    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"]
Jul 24 22:53:39.075100 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.075075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fps4r" event={"ID":"9ad14177-7c17-4970-97e9-f22e67ee5712","Type":"ContainerStarted","Data":"47b05352537c80a0f5f54d56dea02e1c72615b59930689e07e44ab9820991667"}
Jul 24 22:53:39.076638 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.076618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6" event={"ID":"f7bcc154-1403-44bb-99b5-b0d2c7d23168","Type":"ContainerStarted","Data":"12889127fb0953b61e100fd6ff04e87c521a2c8168f4280640f70f634c1a7cc7"}
Jul 24 22:53:39.078245 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.078222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d" event={"ID":"bd52e67e-d081-4e6f-93b8-5579f43a4537","Type":"ContainerStarted","Data":"b743348ecb4f274e3dac6a5719fb91d9f0cd03a5ce822bc75395e29e65af0e5c"}
Jul 24 22:53:39.093905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.093860    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fps4r" podStartSLOduration=17.078571213 podStartE2EDuration="33.093847866s" totalImagesPullingTime="16.015276653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:53:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:39.092605294 +0000 UTC m=+79.945931844" watchObservedRunningTime="2026-07-24 22:53:39.093847866 +0000 UTC m=+79.947174406"
Jul 24 22:53:39.108000 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.107960    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-h2xp6" podStartSLOduration=34.236228257 podStartE2EDuration="38.107948965s" totalImagesPullingTime="3.871720708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:39.107441606 +0000 UTC m=+79.960768147" watchObservedRunningTime="2026-07-24 22:53:39.107948965 +0000 UTC m=+79.961275501"
Jul 24 22:53:39.122816 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.122776    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4fl6d" podStartSLOduration=34.229364484 podStartE2EDuration="38.122767838s" totalImagesPullingTime="3.893403354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:39.121652803 +0000 UTC m=+79.974979345" watchObservedRunningTime="2026-07-24 22:53:39.122767838 +0000 UTC m=+79.976094392"
Jul 24 22:53:39.141416 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.141368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a9859ad3-0bc1-4df1-9da3-361e6b614f23-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hm7jj\" (UID: \"a9859ad3-0bc1-4df1-9da3-361e6b614f23\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:39.204097 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.204064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.242169 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.242141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a9859ad3-0bc1-4df1-9da3-361e6b614f23-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hm7jj\" (UID: \"a9859ad3-0bc1-4df1-9da3-361e6b614f23\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:39.242284 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:39.242264    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 22:53:39.242346 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:39.242332    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9859ad3-0bc1-4df1-9da3-361e6b614f23-tls-certificates podName:a9859ad3-0bc1-4df1-9da3-361e6b614f23 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:39.742311566 +0000 UTC m=+80.595638104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a9859ad3-0bc1-4df1-9da3-361e6b614f23-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-hm7jj" (UID: "a9859ad3-0bc1-4df1-9da3-361e6b614f23") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 22:53:39.305718 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.305688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.361068 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.361032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.736060 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.736027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.746980 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.746957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a9859ad3-0bc1-4df1-9da3-361e6b614f23-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hm7jj\" (UID: \"a9859ad3-0bc1-4df1-9da3-361e6b614f23\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:39.749228 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.749207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a9859ad3-0bc1-4df1-9da3-361e6b614f23-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hm7jj\" (UID: \"a9859ad3-0bc1-4df1-9da3-361e6b614f23\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:39.855338 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.855308    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.909049 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.909018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:39.953035 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:39.953013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:40.072007 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:40.071981    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"]
Jul 24 22:53:40.074209 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:40.074182    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9859ad3_0bc1_4df1_9da3_361e6b614f23.slice/crio-73ed0d6887bdc56d09422674919a2aaa3d2c9d25b2738b8d71771481c744356d WatchSource:0}: Error finding container 73ed0d6887bdc56d09422674919a2aaa3d2c9d25b2738b8d71771481c744356d: Status 404 returned error can't find the container with id 73ed0d6887bdc56d09422674919a2aaa3d2c9d25b2738b8d71771481c744356d
Jul 24 22:53:40.083170 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:40.083147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj" event={"ID":"a9859ad3-0bc1-4df1-9da3-361e6b614f23","Type":"ContainerStarted","Data":"73ed0d6887bdc56d09422674919a2aaa3d2c9d25b2738b8d71771481c744356d"}
Jul 24 22:53:40.963869 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:40.963835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:41.018038 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:41.018008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:42.092575 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:42.092541    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj" event={"ID":"a9859ad3-0bc1-4df1-9da3-361e6b614f23","Type":"ContainerStarted","Data":"5ea06399b2adfd91eb408c5dc2daa8ac929e0462c8defd5fae1a8e00fd3a71e5"}
Jul 24 22:53:42.092932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:42.092660    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:42.097173 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:42.097155    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj"
Jul 24 22:53:42.107539 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:42.107490    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hm7jj" podStartSLOduration=1.881200646 podStartE2EDuration="3.107476644s" totalImagesPullingTime="1.226275998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:42.106018737 +0000 UTC m=+82.959345276" watchObservedRunningTime="2026-07-24 22:53:42.107476644 +0000 UTC m=+82.960803185"
Jul 24 22:53:43.065693 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:43.065657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:43.135037 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:43.135003    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:43.390268 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:43.390236    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:43.689146 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:43.689063    2564 scope.go:117] "RemoveContainer" containerID="56c42b4ab2a84d0b2295e61ff10c9160f5ab6febdca03bc27787bc89fdf80a10"
Jul 24 22:53:44.101407 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.101384    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 22:53:44.101539 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.101501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" event={"ID":"232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3","Type":"ContainerStarted","Data":"ecce1e845ddfe0b6a0d7f09d213996ac2a2038dfa3560e310ca79c810626d0ce"}
Jul 24 22:53:44.101808 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.101779    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:44.106246 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.106225    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz"
Jul 24 22:53:44.117687 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.117649    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-8qwwz" podStartSLOduration=38.676526639 podStartE2EDuration="43.117637902s" totalImagesPullingTime="4.441111263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:44.117373073 +0000 UTC m=+84.970699612" watchObservedRunningTime="2026-07-24 22:53:44.117637902 +0000 UTC m=+84.970964441"
Jul 24 22:53:44.199107 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.199082    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-2sngs"]
Jul 24 22:53:44.202034 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.202016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:53:44.206419 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.206399    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 22:53:44.206514 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.206404    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jpln2\""
Jul 24 22:53:44.206514 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.206423    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 22:53:44.212471 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.212451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-2sngs"]
Jul 24 22:53:44.221141 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.221101    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:44.284960 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.284938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfh4c\" (UniqueName: \"kubernetes.io/projected/ac9b9a00-cb92-46cb-b57f-fd50c71e1d46-kube-api-access-hfh4c\") pod \"downloads-7c97dc4555-2sngs\" (UID: \"ac9b9a00-cb92-46cb-b57f-fd50c71e1d46\") " pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:53:44.287508 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.287486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:44.338261 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.338240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:44.386108 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.386058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfh4c\" (UniqueName: \"kubernetes.io/projected/ac9b9a00-cb92-46cb-b57f-fd50c71e1d46-kube-api-access-hfh4c\") pod \"downloads-7c97dc4555-2sngs\" (UID: \"ac9b9a00-cb92-46cb-b57f-fd50c71e1d46\") " pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:53:44.393788 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.393761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfh4c\" (UniqueName: \"kubernetes.io/projected/ac9b9a00-cb92-46cb-b57f-fd50c71e1d46-kube-api-access-hfh4c\") pod \"downloads-7c97dc4555-2sngs\" (UID: \"ac9b9a00-cb92-46cb-b57f-fd50c71e1d46\") " pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:53:44.502023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.501995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:44.510914 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.510892    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:53:44.632225 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.632193    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-2sngs"]
Jul 24 22:53:44.635208 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:44.635180    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9b9a00_cb92_46cb_b57f_fd50c71e1d46.slice/crio-42afe8126440addcfe905a52c512a24d9be17a821ead129ed20e127ad0b4e84b WatchSource:0}: Error finding container 42afe8126440addcfe905a52c512a24d9be17a821ead129ed20e127ad0b4e84b: Status 404 returned error can't find the container with id 42afe8126440addcfe905a52c512a24d9be17a821ead129ed20e127ad0b4e84b
Jul 24 22:53:44.883606 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.883576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:44.939207 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:44.939119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:45.064647 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.064623    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pqdb6"
Jul 24 22:53:45.115975 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.115943    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-2sngs" event={"ID":"ac9b9a00-cb92-46cb-b57f-fd50c71e1d46","Type":"ContainerStarted","Data":"42afe8126440addcfe905a52c512a24d9be17a821ead129ed20e127ad0b4e84b"}
Jul 24 22:53:45.195993 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.195910    2564 patch_prober.go:28] interesting pod/image-registry-c6bcc8959-5xkt9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 22:53:45.195993 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.195972    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" podUID="707db365-5d14-4f25-b7b2-1bcc92208922" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 22:53:45.279568 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.279532    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:45.332867 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.332830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:45.690349 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.690255    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:45.744987 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.744958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:45.887176 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.887140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:45.941271 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:45.941161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.270929 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.270885    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.355557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.355519    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.462258 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.462221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.470116 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.470088    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"]
Jul 24 22:53:47.473800 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.473771    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.476359 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.476331    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 24 22:53:47.476359 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.476347    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 24 22:53:47.476701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.476645    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9kv2q\""
Jul 24 22:53:47.476981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.476962    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 22:53:47.484966 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.484945    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"]
Jul 24 22:53:47.492415 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.492395    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"]
Jul 24 22:53:47.495686 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.495667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.497856 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.497830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 22:53:47.497940 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.497862    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kj7qh\""
Jul 24 22:53:47.498008 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.497960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 22:53:47.498304 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.498286    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 22:53:47.507364 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.507328    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qhrdg"]
Jul 24 22:53:47.510780 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.510758    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"]
Jul 24 22:53:47.510892 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.510862    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.512814 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.512792    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 22:53:47.513022 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.513003    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 22:53:47.513144 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.513069    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xbwd8\""
Jul 24 22:53:47.513352 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.513325    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 22:53:47.550660 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.550593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.613379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.613503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.613503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613420    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzmrv\" (UniqueName: \"kubernetes.io/projected/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-kube-api-access-hzmrv\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.613503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-root\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.613503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.613704 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-sys\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.613704 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613651    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8ef29862-0e6d-439c-8b64-f26d3f42e096-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.613704 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p59ww\" (UniqueName: \"kubernetes.io/projected/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-api-access-p59ww\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.613856 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef29862-0e6d-439c-8b64-f26d3f42e096-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.613856 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f9141c0-c865-4c73-9805-09e3fba95010-metrics-client-ca\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.613856 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-wtmp\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.614003 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613866    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-tls\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.614003 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x8j6\" (UniqueName: \"kubernetes.io/projected/7f9141c0-c865-4c73-9805-09e3fba95010-kube-api-access-4x8j6\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.614003 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.613974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-accelerators-collector-config\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.614165 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.614005    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.614165 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.614029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-textfile\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.614165 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.614075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.614165 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.614115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.614308 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.614173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.615231 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.615195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.714520 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-sys\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.714664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8ef29862-0e6d-439c-8b64-f26d3f42e096-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.714664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p59ww\" (UniqueName: \"kubernetes.io/projected/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-api-access-p59ww\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.714664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef29862-0e6d-439c-8b64-f26d3f42e096-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.714664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f9141c0-c865-4c73-9805-09e3fba95010-metrics-client-ca\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.714664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-wtmp\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.714664 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-tls\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x8j6\" (UniqueName: \"kubernetes.io/projected/7f9141c0-c865-4c73-9805-09e3fba95010-kube-api-access-4x8j6\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-accelerators-collector-config\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-textfile\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714935    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8ef29862-0e6d-439c-8b64-f26d3f42e096-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.715074 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.714601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-sys\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f9141c0-c865-4c73-9805-09e3fba95010-metrics-client-ca\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715325    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-textfile\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715387    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ef29862-0e6d-439c-8b64-f26d3f42e096-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.715509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715479    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-accelerators-collector-config\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715702 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.715702 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-wtmp\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715702 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:47.715654    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 24 22:53:47.715702 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715981 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:47.715710    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-tls podName:f12447d0-4fd8-49df-b6f5-2ac3894cb31f nodeName:}" failed. No retries permitted until 2026-07-24 22:53:48.215691084 +0000 UTC m=+89.069017606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-p29hm" (UID: "f12447d0-4fd8-49df-b6f5-2ac3894cb31f") : secret "openshift-state-metrics-tls" not found
Jul 24 22:53:47.715981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.715981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715791    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzmrv\" (UniqueName: \"kubernetes.io/projected/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-kube-api-access-hzmrv\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.715981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-root\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.715981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.715981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.715842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.716297 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.716172    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f9141c0-c865-4c73-9805-09e3fba95010-root\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.716573 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.716548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.717917 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.717867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.718398 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.718368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.718515 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.718495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.718913 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.718871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f9141c0-c865-4c73-9805-09e3fba95010-node-exporter-tls\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.719096 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.719070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.719318 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.719261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.723516 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.723491    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x8j6\" (UniqueName: \"kubernetes.io/projected/7f9141c0-c865-4c73-9805-09e3fba95010-kube-api-access-4x8j6\") pod \"node-exporter-qhrdg\" (UID: \"7f9141c0-c865-4c73-9805-09e3fba95010\") " pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.723630 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.723608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p59ww\" (UniqueName: \"kubernetes.io/projected/8ef29862-0e6d-439c-8b64-f26d3f42e096-kube-api-access-p59ww\") pod \"kube-state-metrics-56d9f944b4-4v4mn\" (UID: \"8ef29862-0e6d-439c-8b64-f26d3f42e096\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.725796 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.725737    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzmrv\" (UniqueName: \"kubernetes.io/projected/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-kube-api-access-hzmrv\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:47.797793 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.797534    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.806718 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.806654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"
Jul 24 22:53:47.821870 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.821847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qhrdg"
Jul 24 22:53:47.836170 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:47.836139    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9141c0_c865_4c73_9805_09e3fba95010.slice/crio-a953e4567cbb59859e632073957fb63f1c72958aad703e038d696e41da5a1eaa WatchSource:0}: Error finding container a953e4567cbb59859e632073957fb63f1c72958aad703e038d696e41da5a1eaa: Status 404 returned error can't find the container with id a953e4567cbb59859e632073957fb63f1c72958aad703e038d696e41da5a1eaa
Jul 24 22:53:47.862520 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.862451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.952335 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.952302    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn"]
Jul 24 22:53:47.955031 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:47.955002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:47.955768 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:47.955745    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef29862_0e6d_439c_8b64_f26d3f42e096.slice/crio-dd01dc2fdb71490b1094346c1b67bbd1a0e74198dc0d6d178035b3eade5d16d4 WatchSource:0}: Error finding container dd01dc2fdb71490b1094346c1b67bbd1a0e74198dc0d6d178035b3eade5d16d4: Status 404 returned error can't find the container with id dd01dc2fdb71490b1094346c1b67bbd1a0e74198dc0d6d178035b3eade5d16d4
Jul 24 22:53:48.010911 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.010879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:48.129238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.129195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhrdg" event={"ID":"7f9141c0-c865-4c73-9805-09e3fba95010","Type":"ContainerStarted","Data":"a953e4567cbb59859e632073957fb63f1c72958aad703e038d696e41da5a1eaa"}
Jul 24 22:53:48.130415 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.130391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn" event={"ID":"8ef29862-0e6d-439c-8b64-f26d3f42e096","Type":"ContainerStarted","Data":"dd01dc2fdb71490b1094346c1b67bbd1a0e74198dc0d6d178035b3eade5d16d4"}
Jul 24 22:53:48.220091 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.220050    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:48.222816 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.222791    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12447d0-4fd8-49df-b6f5-2ac3894cb31f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-p29hm\" (UID: \"f12447d0-4fd8-49df-b6f5-2ac3894cb31f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:48.388745 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.388662    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"
Jul 24 22:53:48.440782 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.440756    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:48.533673 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.533614    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:48.535647 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.535617    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:53:48.541442 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.541340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.544246 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544068    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 22:53:48.544246 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544108    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 22:53:48.544246 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544139    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 22:53:48.544246 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544222    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 22:53:48.544497 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544419    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 22:53:48.544497 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544432    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 22:53:48.544743 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544722    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 22:53:48.544829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544773    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 22:53:48.545007 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.544989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bmjbz\""
Jul 24 22:53:48.545285 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.545266    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 22:53:48.560668 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.559951    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm"]
Jul 24 22:53:48.560668 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.560357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:53:48.560847 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:48.560812    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12447d0_4fd8_49df_b6f5_2ac3894cb31f.slice/crio-2271d4c00dae9269837bab54f07c524f2fe46805b972c54b632657e1f193786d WatchSource:0}: Error finding container 2271d4c00dae9269837bab54f07c524f2fe46805b972c54b632657e1f193786d: Status 404 returned error can't find the container with id 2271d4c00dae9269837bab54f07c524f2fe46805b972c54b632657e1f193786d
Jul 24 22:53:48.724351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6k9\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-kube-api-access-gt6k9\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724341    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-config-out\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-config-volume\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-web-config\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724494    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724558    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.724728 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.724607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825193    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-config-out\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-config-volume\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-web-config\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6k9\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-kube-api-access-gt6k9\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.825978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.825621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.830401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.827833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.830401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.828720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.830401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.830017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.830634 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:48.830521    2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 24 22:53:48.830634 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:53:48.830603    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls podName:96e9fa8d-6e31-44f2-82b4-409627513425 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:49.33058283 +0000 UTC m=+90.183909356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425") : secret "alertmanager-main-tls" not found
Jul 24 22:53:48.832957 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.832914    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-config-out\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.833508 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.833452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.835429 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.833989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.835429 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.834389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-web-config\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.835429 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.834779    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.835429 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.835112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.835612 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.835446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.837372 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.837312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-config-volume\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:48.839564 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:48.839540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6k9\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-kube-api-access-gt6k9\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:49.028979 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.028258    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:53:49.118607 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.118557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:49.137366 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.137215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm" event={"ID":"f12447d0-4fd8-49df-b6f5-2ac3894cb31f","Type":"ContainerStarted","Data":"90735158c0269d6c0eb974418fc9e1ba624ab78e1d5be14e98713b0357ea6631"}
Jul 24 22:53:49.137366 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.137282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm" event={"ID":"f12447d0-4fd8-49df-b6f5-2ac3894cb31f","Type":"ContainerStarted","Data":"d6cd08252d4a50eaf794a8c6ff9b742cc9cb02b7e5817a492f04d0b41ba6fe5d"}
Jul 24 22:53:49.137366 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.137310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm" event={"ID":"f12447d0-4fd8-49df-b6f5-2ac3894cb31f","Type":"ContainerStarted","Data":"2271d4c00dae9269837bab54f07c524f2fe46805b972c54b632657e1f193786d"}
Jul 24 22:53:49.139533 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.139226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhrdg" event={"ID":"7f9141c0-c865-4c73-9805-09e3fba95010","Type":"ContainerStarted","Data":"0235eb9907053c9d34633feeb1b44dc9e2f4a15cff257ba92c4576d4abdbf138"}
Jul 24 22:53:49.331886 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.331845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:49.334922 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.334893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:49.460776 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.460646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:53:49.511865 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.511818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:49.611912 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.611834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:49.724068 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.724045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:53:49.728997 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:53:49.728919    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96e9fa8d_6e31_44f2_82b4_409627513425.slice/crio-ee66739e9d98bd6d01e62587eb6fe803ac5df361891b0411101c6d4897f1c535 WatchSource:0}: Error finding container ee66739e9d98bd6d01e62587eb6fe803ac5df361891b0411101c6d4897f1c535: Status 404 returned error can't find the container with id ee66739e9d98bd6d01e62587eb6fe803ac5df361891b0411101c6d4897f1c535
Jul 24 22:53:49.734627 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.733712    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c6bcc8959-5xkt9"]
Jul 24 22:53:49.877709 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.877653    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:49.978228 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:49.978179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:50.148192 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.147852    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn" event={"ID":"8ef29862-0e6d-439c-8b64-f26d3f42e096","Type":"ContainerStarted","Data":"2a7fb76fc60963185e065fef53a034d38f6a807f1f945ba93b2f5b4f5809f732"}
Jul 24 22:53:50.148192 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.147952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn" event={"ID":"8ef29862-0e6d-439c-8b64-f26d3f42e096","Type":"ContainerStarted","Data":"cd61346e5b1f90f100da0992a358538778c006bc65619fa99d6c305f96d1763b"}
Jul 24 22:53:50.148192 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.147982    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn" event={"ID":"8ef29862-0e6d-439c-8b64-f26d3f42e096","Type":"ContainerStarted","Data":"731955cc0f8696dfe02f9bdf2c4a87bade0574094fef0c527f37abef07709c73"}
Jul 24 22:53:50.150249 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.150217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"ee66739e9d98bd6d01e62587eb6fe803ac5df361891b0411101c6d4897f1c535"}
Jul 24 22:53:50.152585 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.152557    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f9141c0-c865-4c73-9805-09e3fba95010" containerID="0235eb9907053c9d34633feeb1b44dc9e2f4a15cff257ba92c4576d4abdbf138" exitCode=0
Jul 24 22:53:50.152752 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.152626    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhrdg" event={"ID":"7f9141c0-c865-4c73-9805-09e3fba95010","Type":"ContainerDied","Data":"0235eb9907053c9d34633feeb1b44dc9e2f4a15cff257ba92c4576d4abdbf138"}
Jul 24 22:53:50.185214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.185156    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4v4mn" podStartSLOduration=1.5161844389999999 podStartE2EDuration="3.185134213s" totalImagesPullingTime="1.668949774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:50.165209032 +0000 UTC m=+91.018535574" watchObservedRunningTime="2026-07-24 22:53:50.185134213 +0000 UTC m=+91.038460746"
Jul 24 22:53:50.459360 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.459116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:50.519586 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.519552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:50.581385 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.581196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:50.697081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:50.696988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:51.110724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.110687    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:51.160309 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.160263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhrdg" event={"ID":"7f9141c0-c865-4c73-9805-09e3fba95010","Type":"ContainerStarted","Data":"607fcad25184d1322e3846926a1c13611cb05768c70fbf2a5a1e12d3e6a5b838"}
Jul 24 22:53:51.160479 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.160352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhrdg" event={"ID":"7f9141c0-c865-4c73-9805-09e3fba95010","Type":"ContainerStarted","Data":"a86579be39bafa2c9eedfb0c618bd14fa858b47e364e9803265716e0f8bf65fa"}
Jul 24 22:53:51.163181 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.163154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm" event={"ID":"f12447d0-4fd8-49df-b6f5-2ac3894cb31f","Type":"ContainerStarted","Data":"78277c827a614873464006890df781692c10c91bb7d130b20ec79127acc5153e"}
Jul 24 22:53:51.177678 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.177629    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qhrdg" podStartSLOduration=2.923067919 podStartE2EDuration="4.177611083s" totalImagesPullingTime="1.170533114s" totalInitContainerRuntime="84.01005ms" isStatefulPod=true podCreationTimestamp="2026-07-24 22:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:51.176793675 +0000 UTC m=+92.030120243" watchObservedRunningTime="2026-07-24 22:53:51.177611083 +0000 UTC m=+92.030937625"
Jul 24 22:53:51.181175 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.181147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:51.192890 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:51.192545    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-p29hm" podStartSLOduration=2.839622956 podStartE2EDuration="4.192526611s" totalImagesPullingTime="1.352903655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:51.192269348 +0000 UTC m=+92.045595889" watchObservedRunningTime="2026-07-24 22:53:51.192526611 +0000 UTC m=+92.045853155"
Jul 24 22:53:52.169288 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:52.169248    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d" exitCode=0
Jul 24 22:53:52.169736 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:52.169338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"}
Jul 24 22:53:52.232953 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:52.232903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:52.293442 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:52.293406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:52.939862 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:52.939766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:53.764356 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:53.764328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:54.187302 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.187221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:54.188450 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.188424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"}
Jul 24 22:53:54.188588 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.188527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"}
Jul 24 22:53:54.188588 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.188569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"}
Jul 24 22:53:54.188701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.188607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"}
Jul 24 22:53:54.188701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.188648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"}
Jul 24 22:53:54.241414 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.241386    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:54.859081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.859042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:54.908340 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:54.908310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:57.104252 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:57.104201    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:57.205542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:57.205464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:59.716952 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:59.716870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:53:59.768902 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:53:59.768867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:00.033522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:00.033448    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jlnvw"
Jul 24 22:54:02.224809 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.224776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-2sngs" event={"ID":"ac9b9a00-cb92-46cb-b57f-fd50c71e1d46","Type":"ContainerStarted","Data":"6c1641df8c34bd03fd842d25afe5b3c8f0fa35041387df69370a120b481a8026"}
Jul 24 22:54:02.225257 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.224842    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:54:02.226511 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.226490    2564 patch_prober.go:28] interesting pod/downloads-7c97dc4555-2sngs container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused" start-of-body=
Jul 24 22:54:02.226622 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.226530    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-2sngs" podUID="ac9b9a00-cb92-46cb-b57f-fd50c71e1d46" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused"
Jul 24 22:54:02.228355 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.228332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerStarted","Data":"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"}
Jul 24 22:54:02.241672 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.241631    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-2sngs" podStartSLOduration=0.793041087 podStartE2EDuration="18.241618646s" totalImagesPullingTime="17.448577559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:02.241041429 +0000 UTC m=+103.094367969" watchObservedRunningTime="2026-07-24 22:54:02.241618646 +0000 UTC m=+103.094945234"
Jul 24 22:54:02.266291 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:02.266241    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.495966566 podStartE2EDuration="14.266222589s" totalImagesPullingTime="10.754000011s" totalInitContainerRuntime="16.256012ms" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:02.265921836 +0000 UTC m=+103.119248389" watchObservedRunningTime="2026-07-24 22:54:02.266222589 +0000 UTC m=+103.119549130"
Jul 24 22:54:03.183734 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:03.183680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:03.245018 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:03.244993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-2sngs"
Jul 24 22:54:03.269497 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:03.269462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:03.322271 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:03.322240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:04.357365 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.357330    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b5c45d-lcgvg"]
Jul 24 22:54:04.390791 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.390753    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b5c45d-lcgvg"]
Jul 24 22:54:04.390941 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.390874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.393805 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.393780    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 22:54:04.393805 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.393780    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 22:54:04.393994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.393856    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 22:54:04.393994 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.393866    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 22:54:04.394467 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.394442    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fjmtg\""
Jul 24 22:54:04.394574 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.394468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 22:54:04.398632 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.398608    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 22:54:04.462774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.462746    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-oauth-config\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.462951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.462792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-trusted-ca-bundle\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.462951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.462850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-service-ca\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.462951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.462881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-config\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.462951 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.462927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wlgt\" (UniqueName: \"kubernetes.io/projected/f4ed7f03-09a3-4752-b963-fe1add95dadc-kube-api-access-5wlgt\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.463173 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.463021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-serving-cert\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.463173 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.463053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-oauth-serving-cert\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564463 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wlgt\" (UniqueName: \"kubernetes.io/projected/f4ed7f03-09a3-4752-b963-fe1add95dadc-kube-api-access-5wlgt\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564637 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-serving-cert\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564637 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564512    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-oauth-serving-cert\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564637 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-oauth-config\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564637 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-trusted-ca-bundle\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564637 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-service-ca\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.564911 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.564660    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-config\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.565420 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.565392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-config\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.565522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.565392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-oauth-serving-cert\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.565603 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.565578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-service-ca\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.565776 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.565634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-trusted-ca-bundle\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.567386 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.567359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-oauth-config\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.567810 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.567792    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-serving-cert\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.572986 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.572964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wlgt\" (UniqueName: \"kubernetes.io/projected/f4ed7f03-09a3-4752-b963-fe1add95dadc-kube-api-access-5wlgt\") pod \"console-7b5c45d-lcgvg\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") " pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.702620 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.702547    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:04.887299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:04.887148    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b5c45d-lcgvg"]
Jul 24 22:54:04.890272 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:54:04.890235    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ed7f03_09a3_4752_b963_fe1add95dadc.slice/crio-760aa756cbf3200cfbf1b5ccd6ebe8e58326aa2e78a4abb8cc9d4084a5913130 WatchSource:0}: Error finding container 760aa756cbf3200cfbf1b5ccd6ebe8e58326aa2e78a4abb8cc9d4084a5913130: Status 404 returned error can't find the container with id 760aa756cbf3200cfbf1b5ccd6ebe8e58326aa2e78a4abb8cc9d4084a5913130
Jul 24 22:54:05.242504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:05.242467    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5c45d-lcgvg" event={"ID":"f4ed7f03-09a3-4752-b963-fe1add95dadc","Type":"ContainerStarted","Data":"760aa756cbf3200cfbf1b5ccd6ebe8e58326aa2e78a4abb8cc9d4084a5913130"}
Jul 24 22:54:05.911413 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:05.911361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:05.964278 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:05.964194    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:07.523262 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:07.523230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:07.619530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:07.619498    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:08.153918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:08.153855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:08.255774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:08.255734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:09.264460 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:09.264401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5c45d-lcgvg" event={"ID":"f4ed7f03-09a3-4752-b963-fe1add95dadc","Type":"ContainerStarted","Data":"aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219"}
Jul 24 22:54:09.284558 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:09.284502    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b5c45d-lcgvg" podStartSLOduration=1.694017891 podStartE2EDuration="5.284479737s" totalImagesPullingTime="3.590461846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:09.282801964 +0000 UTC m=+110.136128515" watchObservedRunningTime="2026-07-24 22:54:09.284479737 +0000 UTC m=+110.137806277"
Jul 24 22:54:09.875916 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:09.875876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:09.975540 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:09.975512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:12.349626 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:12.349594    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:12.450371 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:12.450342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:14.702673 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:14.702633    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:14.703210 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:14.702693    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:14.707602 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:14.707581    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:14.766807 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:14.766762    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" podUID="707db365-5d14-4f25-b7b2-1bcc92208922" containerName="registry" containerID="cri-o://0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4" gracePeriod=30
Jul 24 22:54:15.049893 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.049670    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:54:15.161158 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161114    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-bound-sa-token\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161165    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-trusted-ca\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161194    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161215    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-installation-pull-secrets\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161263    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/707db365-5d14-4f25-b7b2-1bcc92208922-ca-trust-extracted\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161298    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-image-registry-private-configuration\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161560 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161326    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-registry-certificates\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.161560 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161364    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrgsj\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-kube-api-access-lrgsj\") pod \"707db365-5d14-4f25-b7b2-1bcc92208922\" (UID: \"707db365-5d14-4f25-b7b2-1bcc92208922\") "
Jul 24 22:54:15.163259 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161718    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:54:15.163259 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.161824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:54:15.163756 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.163730    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:54:15.163878 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.163856    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:54:15.163938 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.163872    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-kube-api-access-lrgsj" (OuterVolumeSpecName: "kube-api-access-lrgsj") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "kube-api-access-lrgsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:54:15.163938 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.163912    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:54:15.164063 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.164047    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:54:15.170418 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.170394    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707db365-5d14-4f25-b7b2-1bcc92208922-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "707db365-5d14-4f25-b7b2-1bcc92208922" (UID: "707db365-5d14-4f25-b7b2-1bcc92208922"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:54:15.262778 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262710    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrgsj\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-kube-api-access-lrgsj\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.262778 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262735    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-bound-sa-token\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.262778 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262751    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-trusted-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.262778 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262762    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/707db365-5d14-4f25-b7b2-1bcc92208922-registry-tls\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.262778 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262775    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-installation-pull-secrets\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.263057 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262790    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/707db365-5d14-4f25-b7b2-1bcc92208922-ca-trust-extracted\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.263057 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262802    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/707db365-5d14-4f25-b7b2-1bcc92208922-image-registry-private-configuration\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.263057 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.262816    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/707db365-5d14-4f25-b7b2-1bcc92208922-registry-certificates\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:54:15.289359 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.289330    2564 generic.go:356] "Generic (PLEG): container finished" podID="707db365-5d14-4f25-b7b2-1bcc92208922" containerID="0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4" exitCode=0
Jul 24 22:54:15.289473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.289377    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9"
Jul 24 22:54:15.289473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.289425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" event={"ID":"707db365-5d14-4f25-b7b2-1bcc92208922","Type":"ContainerDied","Data":"0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4"}
Jul 24 22:54:15.289582 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.289482    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcc8959-5xkt9" event={"ID":"707db365-5d14-4f25-b7b2-1bcc92208922","Type":"ContainerDied","Data":"033896cc68e4789b90fd75c30a636e1bb864d11e52ac1ac88f0a812467a4943a"}
Jul 24 22:54:15.289582 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.289511    2564 scope.go:117] "RemoveContainer" containerID="0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4"
Jul 24 22:54:15.293532 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.293515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:54:15.298099 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.298086    2564 scope.go:117] "RemoveContainer" containerID="0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4"
Jul 24 22:54:15.298360 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:54:15.298342    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4\": container with ID starting with 0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4 not found: ID does not exist" containerID="0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4"
Jul 24 22:54:15.298405 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.298368    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4"} err="failed to get container status \"0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4\": rpc error: code = NotFound desc = could not find container \"0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4\": container with ID starting with 0cc737660b94cd949b16522a10314543217cf1e0a192d5a0b746b9bee2fa85b4 not found: ID does not exist"
Jul 24 22:54:15.326760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.326732    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c6bcc8959-5xkt9"]
Jul 24 22:54:15.337518 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.337499    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c6bcc8959-5xkt9"]
Jul 24 22:54:15.693042 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:15.693009    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="707db365-5d14-4f25-b7b2-1bcc92208922" path="/var/lib/kubelet/pods/707db365-5d14-4f25-b7b2-1bcc92208922/volumes"
Jul 24 22:54:16.919697 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:16.919660    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:16.973019 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:16.972985    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:17.148459 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:17.148423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:17.248032 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:17.247962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:17.348543 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:17.348509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:17.400990 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:17.400959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:17.716413 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:17.716388    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:17.769874 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:17.769846    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:22.318751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:22.318674    2564 generic.go:356] "Generic (PLEG): container finished" podID="94ea935e-eeaf-4f1c-9acb-8e108377860a" containerID="c51430bca81ec04c347c2269143c0f0063c45f2a9f13ebfb1b13734e3d8b22cc" exitCode=0
Jul 24 22:54:22.318751 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:22.318737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88" event={"ID":"94ea935e-eeaf-4f1c-9acb-8e108377860a","Type":"ContainerDied","Data":"c51430bca81ec04c347c2269143c0f0063c45f2a9f13ebfb1b13734e3d8b22cc"}
Jul 24 22:54:22.319176 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:22.319034    2564 scope.go:117] "RemoveContainer" containerID="c51430bca81ec04c347c2269143c0f0063c45f2a9f13ebfb1b13734e3d8b22cc"
Jul 24 22:54:23.324008 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:23.323977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-sxx88" event={"ID":"94ea935e-eeaf-4f1c-9acb-8e108377860a","Type":"ContainerStarted","Data":"d72fd8557402f478635afefb648364563b21409c8b843dd20cdf3fb8926076b7"}
Jul 24 22:54:24.862931 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:24.862891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:24.917472 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:24.917447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:39.856846 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:39.856817    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:54:39.908659 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:54:39.908630    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:02.604594 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:02.604552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:03.623385 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:03.623348    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:03.719525 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:03.719488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:07.884336 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884282    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:55:07.884722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884608    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="alertmanager" containerID="cri-o://97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a" gracePeriod=120
Jul 24 22:55:07.884722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884673    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-metric" containerID="cri-o://88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9" gracePeriod=120
Jul 24 22:55:07.884722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884682    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-web" containerID="cri-o://c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d" gracePeriod=120
Jul 24 22:55:07.884857 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884704    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="config-reloader" containerID="cri-o://50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa" gracePeriod=120
Jul 24 22:55:07.885190 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884726    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy" containerID="cri-o://c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319" gracePeriod=120
Jul 24 22:55:07.885190 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:07.884754    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="prom-label-proxy" containerID="cri-o://9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7" gracePeriod=120
Jul 24 22:55:08.496858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.496829    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7" exitCode=0
Jul 24 22:55:08.496858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.496853    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319" exitCode=0
Jul 24 22:55:08.496858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.496860    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa" exitCode=0
Jul 24 22:55:08.496858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.496866    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a" exitCode=0
Jul 24 22:55:08.497238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.496900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"}
Jul 24 22:55:08.497238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.496984    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"}
Jul 24 22:55:08.497238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.497011    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"}
Jul 24 22:55:08.497238 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:08.497034    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"}
Jul 24 22:55:09.117734 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.117713    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.172969 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.172945    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-tls-assets\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173086 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.172986    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-config-out\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173086 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173017    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-web\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173086 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173072    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173295 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-trusted-ca-bundle\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173316    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173414 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-web-config\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173478 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173440    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt6k9\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-kube-api-access-gt6k9\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173478 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173465    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-cluster-tls-config\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173588 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173506    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-main-db\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173588 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173536    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-metric\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173588 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-config-volume\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173587    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-metrics-client-ca\") pod \"96e9fa8d-6e31-44f2-82b4-409627513425\" (UID: \"96e9fa8d-6e31-44f2-82b4-409627513425\") "
Jul 24 22:55:09.173724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173639    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:09.173883 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.173865    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.174307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.174263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:55:09.174479 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.174455    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:09.175812 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.175777    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-config-out" (OuterVolumeSpecName: "config-out") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:55:09.176677 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.176365    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:55:09.177679 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.177638    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.177952 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.177925    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-config-volume" (OuterVolumeSpecName: "config-volume") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.178635 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.178611    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.179000 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.178981    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.179362 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.179338    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.179458 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.179393    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-kube-api-access-gt6k9" (OuterVolumeSpecName: "kube-api-access-gt6k9") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "kube-api-access-gt6k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:55:09.183687 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.183655    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.191089 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.191060    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-web-config" (OuterVolumeSpecName: "web-config") pod "96e9fa8d-6e31-44f2-82b4-409627513425" (UID: "96e9fa8d-6e31-44f2-82b4-409627513425"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274692    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-alertmanager-main-db\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274716    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274727    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-config-volume\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274735    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96e9fa8d-6e31-44f2-82b4-409627513425-metrics-client-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274743    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-tls-assets\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274751    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96e9fa8d-6e31-44f2-82b4-409627513425-config-out\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.274760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274758    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.275058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274766    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-main-tls\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.275058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274777    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.275058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274785    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-web-config\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.275058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274792    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt6k9\" (UniqueName: \"kubernetes.io/projected/96e9fa8d-6e31-44f2-82b4-409627513425-kube-api-access-gt6k9\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.275058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.274800    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96e9fa8d-6e31-44f2-82b4-409627513425-cluster-tls-config\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:55:09.503204 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503174    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9" exitCode=0
Jul 24 22:55:09.503204 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503196    2564 generic.go:356] "Generic (PLEG): container finished" podID="96e9fa8d-6e31-44f2-82b4-409627513425" containerID="c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d" exitCode=0
Jul 24 22:55:09.503431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"}
Jul 24 22:55:09.503431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503261    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.503431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"}
Jul 24 22:55:09.503431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96e9fa8d-6e31-44f2-82b4-409627513425","Type":"ContainerDied","Data":"ee66739e9d98bd6d01e62587eb6fe803ac5df361891b0411101c6d4897f1c535"}
Jul 24 22:55:09.503431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.503356    2564 scope.go:117] "RemoveContainer" containerID="9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"
Jul 24 22:55:09.511491 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.511457    2564 scope.go:117] "RemoveContainer" containerID="88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"
Jul 24 22:55:09.517972 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.517957    2564 scope.go:117] "RemoveContainer" containerID="c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"
Jul 24 22:55:09.524658 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.524643    2564 scope.go:117] "RemoveContainer" containerID="c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"
Jul 24 22:55:09.526919 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.526657    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:55:09.530079 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.530058    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:55:09.532262 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.532242    2564 scope.go:117] "RemoveContainer" containerID="50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"
Jul 24 22:55:09.539401 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.539388    2564 scope.go:117] "RemoveContainer" containerID="97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"
Jul 24 22:55:09.545960 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.545946    2564 scope.go:117] "RemoveContainer" containerID="e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"
Jul 24 22:55:09.552377 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.552363    2564 scope.go:117] "RemoveContainer" containerID="9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"
Jul 24 22:55:09.552632 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.552615    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7\": container with ID starting with 9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7 not found: ID does not exist" containerID="9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"
Jul 24 22:55:09.552675 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.552645    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"} err="failed to get container status \"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7\": rpc error: code = NotFound desc = could not find container \"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7\": container with ID starting with 9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7 not found: ID does not exist"
Jul 24 22:55:09.552675 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.552662    2564 scope.go:117] "RemoveContainer" containerID="88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"
Jul 24 22:55:09.552881 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.552856    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9\": container with ID starting with 88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9 not found: ID does not exist" containerID="88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"
Jul 24 22:55:09.552937 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.552884    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"} err="failed to get container status \"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9\": rpc error: code = NotFound desc = could not find container \"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9\": container with ID starting with 88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9 not found: ID does not exist"
Jul 24 22:55:09.552937 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.552897    2564 scope.go:117] "RemoveContainer" containerID="c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"
Jul 24 22:55:09.553223 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.553117    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319\": container with ID starting with c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319 not found: ID does not exist" containerID="c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"
Jul 24 22:55:09.553285 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.553230    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"} err="failed to get container status \"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319\": rpc error: code = NotFound desc = could not find container \"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319\": container with ID starting with c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319 not found: ID does not exist"
Jul 24 22:55:09.553285 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.553246    2564 scope.go:117] "RemoveContainer" containerID="c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"
Jul 24 22:55:09.553513 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.553497    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d\": container with ID starting with c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d not found: ID does not exist" containerID="c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"
Jul 24 22:55:09.553576 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.553521    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"} err="failed to get container status \"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d\": rpc error: code = NotFound desc = could not find container \"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d\": container with ID starting with c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d not found: ID does not exist"
Jul 24 22:55:09.553576 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.553542    2564 scope.go:117] "RemoveContainer" containerID="50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"
Jul 24 22:55:09.553772 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.553755    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa\": container with ID starting with 50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa not found: ID does not exist" containerID="50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"
Jul 24 22:55:09.553829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.553776    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"} err="failed to get container status \"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa\": rpc error: code = NotFound desc = could not find container \"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa\": container with ID starting with 50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa not found: ID does not exist"
Jul 24 22:55:09.553829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.553791    2564 scope.go:117] "RemoveContainer" containerID="97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"
Jul 24 22:55:09.554011 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.553996    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a\": container with ID starting with 97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a not found: ID does not exist" containerID="97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"
Jul 24 22:55:09.554067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554013    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"} err="failed to get container status \"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a\": rpc error: code = NotFound desc = could not find container \"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a\": container with ID starting with 97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a not found: ID does not exist"
Jul 24 22:55:09.554067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554028    2564 scope.go:117] "RemoveContainer" containerID="e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"
Jul 24 22:55:09.554266 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:55:09.554251    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d\": container with ID starting with e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d not found: ID does not exist" containerID="e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"
Jul 24 22:55:09.554326 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554273    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"} err="failed to get container status \"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d\": rpc error: code = NotFound desc = could not find container \"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d\": container with ID starting with e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d not found: ID does not exist"
Jul 24 22:55:09.554326 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554304    2564 scope.go:117] "RemoveContainer" containerID="9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"
Jul 24 22:55:09.554520 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554501    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7"} err="failed to get container status \"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7\": rpc error: code = NotFound desc = could not find container \"9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7\": container with ID starting with 9ad4d0db3bdfe395d67617ed1df3a2fc40dc2de423f39a1ec3dd93de6cf386b7 not found: ID does not exist"
Jul 24 22:55:09.554566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554521    2564 scope.go:117] "RemoveContainer" containerID="88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"
Jul 24 22:55:09.554709 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554691    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9"} err="failed to get container status \"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9\": rpc error: code = NotFound desc = could not find container \"88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9\": container with ID starting with 88e1d8e32bbc51a9299734c8928d4bd5a60ab0851bf054abee0d11572079cee9 not found: ID does not exist"
Jul 24 22:55:09.554765 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554710    2564 scope.go:117] "RemoveContainer" containerID="c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"
Jul 24 22:55:09.554903 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554882    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319"} err="failed to get container status \"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319\": rpc error: code = NotFound desc = could not find container \"c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319\": container with ID starting with c058f4ea72721cc664d598ac6a2ea46c183ca04df5f7944e71a05b03b3de0319 not found: ID does not exist"
Jul 24 22:55:09.554950 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.554905    2564 scope.go:117] "RemoveContainer" containerID="c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"
Jul 24 22:55:09.555106 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555088    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d"} err="failed to get container status \"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d\": rpc error: code = NotFound desc = could not find container \"c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d\": container with ID starting with c6b8515fd24ff75f4bc48468c53225092daad449dece123da020487096f6073d not found: ID does not exist"
Jul 24 22:55:09.555173 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555107    2564 scope.go:117] "RemoveContainer" containerID="50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"
Jul 24 22:55:09.555341 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555325    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa"} err="failed to get container status \"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa\": rpc error: code = NotFound desc = could not find container \"50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa\": container with ID starting with 50bd00d9a5eb3f2178c9f4b9ad0a7a57c33f6d2334f965f15b5da6d076d677fa not found: ID does not exist"
Jul 24 22:55:09.555388 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555344    2564 scope.go:117] "RemoveContainer" containerID="97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"
Jul 24 22:55:09.555557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555539    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a"} err="failed to get container status \"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a\": rpc error: code = NotFound desc = could not find container \"97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a\": container with ID starting with 97fabccc764b8f90e2a6bbf7bfc0f3d1f61fe8687df63484a33019bf587a8e7a not found: ID does not exist"
Jul 24 22:55:09.555603 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555558    2564 scope.go:117] "RemoveContainer" containerID="e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"
Jul 24 22:55:09.555761 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.555745    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d"} err="failed to get container status \"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d\": rpc error: code = NotFound desc = could not find container \"e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d\": container with ID starting with e8277e0b892a47d9cc4ba2bb8e91d73e20d470404345c0fe9f1f39dd6196714d not found: ID does not exist"
Jul 24 22:55:09.560495 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560477    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:55:09.560731 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560721    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560733    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560742    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="config-reloader"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560747    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="config-reloader"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560755    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="init-config-reloader"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560760    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="init-config-reloader"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560765    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-web"
Jul 24 22:55:09.560774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560770    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-web"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560780    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="alertmanager"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560785    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="alertmanager"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560793    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-metric"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560798    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-metric"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560805    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="prom-label-proxy"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560810    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="prom-label-proxy"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560816    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707db365-5d14-4f25-b7b2-1bcc92208922" containerName="registry"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560821    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="707db365-5d14-4f25-b7b2-1bcc92208922" containerName="registry"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560862    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="config-reloader"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560868    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="alertmanager"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560881    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-web"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560889    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="707db365-5d14-4f25-b7b2-1bcc92208922" containerName="registry"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560895    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560899    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="kube-rbac-proxy-metric"
Jul 24 22:55:09.560998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.560904    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" containerName="prom-label-proxy"
Jul 24 22:55:09.564251 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.564237    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.566720 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.566700    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 22:55:09.566819 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.566700    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 22:55:09.566819 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.566738    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 22:55:09.566819 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.566804    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 22:55:09.567024 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.567008    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bmjbz\""
Jul 24 22:55:09.567088 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.567008    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 22:55:09.567088 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.567031    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 22:55:09.567219 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.567050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 22:55:09.567219 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.567137    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 22:55:09.571659 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.571642    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 22:55:09.580729 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.580710    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:55:09.677126 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677105    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677244 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677244 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677191    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-web-config\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677244 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvc29\" (UniqueName: \"kubernetes.io/projected/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-kube-api-access-gvc29\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-config-out\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677386    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-config-volume\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677399    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.677530 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.677469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.692608 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.692582    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96e9fa8d-6e31-44f2-82b4-409627513425" path="/var/lib/kubelet/pods/96e9fa8d-6e31-44f2-82b4-409627513425/volumes"
Jul 24 22:55:09.777944 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.777881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.777944 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.777917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.777943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-web-config\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.777966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.777997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvc29\" (UniqueName: \"kubernetes.io/projected/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-kube-api-access-gvc29\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778081 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-config-out\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778365 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.778806 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-config-volume\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778906    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.778963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.779014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.779181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.779239 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.779235    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.780945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.780903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.781062 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.781042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.781114 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.781098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-web-config\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.781469 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.781437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.781547 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.781531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.781611 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.781594    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-config-volume\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.781666 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.781651    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.782423 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.782409    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.782815 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.782796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-config-out\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.785352 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.785336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvc29\" (UniqueName: \"kubernetes.io/projected/3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5-kube-api-access-gvc29\") pod \"alertmanager-main-0\" (UID: \"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:09.873351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:09.873334    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 22:55:10.015488 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:10.015461    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 22:55:10.018376 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:55:10.018350    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dff6dd2_d2d5_442a_b4a0_d7a516a0cde5.slice/crio-6431c67fc1187d13dc475c8e34fd5d65b9a5f5e9c7ef975ce5f7fd4f29114c60 WatchSource:0}: Error finding container 6431c67fc1187d13dc475c8e34fd5d65b9a5f5e9c7ef975ce5f7fd4f29114c60: Status 404 returned error can't find the container with id 6431c67fc1187d13dc475c8e34fd5d65b9a5f5e9c7ef975ce5f7fd4f29114c60
Jul 24 22:55:10.512845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:10.512809    2564 generic.go:356] "Generic (PLEG): container finished" podID="3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5" containerID="69989eef9521e8f7e14af771ec97f547e123d39d4c72e14e5f3d61fa1ee8b40e" exitCode=0
Jul 24 22:55:10.513220 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:10.512897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerDied","Data":"69989eef9521e8f7e14af771ec97f547e123d39d4c72e14e5f3d61fa1ee8b40e"}
Jul 24 22:55:10.513220 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:10.513035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"6431c67fc1187d13dc475c8e34fd5d65b9a5f5e9c7ef975ce5f7fd4f29114c60"}
Jul 24 22:55:11.520338 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.520264    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"9e32daa5ed1cc7239cd476ee4b4fede79f8aa7244cf683defdd73b90d592460b"}
Jul 24 22:55:11.520338 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.520327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"26e2efb7610c028a7c0abf9a27939c88f1931f86969242be2797fe8672648a2e"}
Jul 24 22:55:11.520730 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.520365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"9c27775c309ece22a9f1053181177ef49956d5ab396922b9ecd2b1dd4c384b6b"}
Jul 24 22:55:11.520730 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.520401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"bf63c08a6230021637a771ddc73c2f147245509ca6f7fc807323c8fd0b56b265"}
Jul 24 22:55:11.520730 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.520427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"212d3f463e54463cddb1212fee9c4a14a449c4d4b2a4ea4f444e5d1ac2e823c9"}
Jul 24 22:55:11.520730 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.520452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5","Type":"ContainerStarted","Data":"59e3a7ff1bb0b573df776e2fa79cb5543c2f8a7a90209a3fe0606df4563c7451"}
Jul 24 22:55:11.545265 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:11.545222    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.534418079 podStartE2EDuration="2.545197454s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.779375ms" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:11.54502028 +0000 UTC m=+172.398346820" watchObservedRunningTime="2026-07-24 22:55:11.545197454 +0000 UTC m=+172.398523994"
Jul 24 22:55:15.511798 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:15.511767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:15.566796 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:15.566755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:24.866799 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:24.866757    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:24.967168 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:24.967144    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:25.180151 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.180058    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dbf9947f5-t69wd"]
Jul 24 22:55:25.183469 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.183446    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.194887 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.194864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbf9947f5-t69wd"]
Jul 24 22:55:25.236466 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.236437    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:25.295250 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2d5\" (UniqueName: \"kubernetes.io/projected/dda34048-c829-4281-a3a1-178c440038a9-kube-api-access-lx2d5\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.295347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-oauth-config\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.295347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-serving-cert\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.295347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-console-config\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.295347 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-trusted-ca-bundle\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.295542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-oauth-serving-cert\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.295542 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.295444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-service-ca\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.334526 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.334500    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:25.396643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2d5\" (UniqueName: \"kubernetes.io/projected/dda34048-c829-4281-a3a1-178c440038a9-kube-api-access-lx2d5\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.396726 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-oauth-config\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.396726 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-serving-cert\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.396726 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-console-config\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.396726 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396706    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-trusted-ca-bundle\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.396918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-oauth-serving-cert\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.396918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.396762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-service-ca\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.397458 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.397435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-console-config\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.397548 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.397517    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-oauth-serving-cert\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.397645 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.397623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-service-ca\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.397710 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.397676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-trusted-ca-bundle\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.399179 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.399155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-serving-cert\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.399270 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.399244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-oauth-config\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.403868 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.403850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2d5\" (UniqueName: \"kubernetes.io/projected/dda34048-c829-4281-a3a1-178c440038a9-kube-api-access-lx2d5\") pod \"console-7dbf9947f5-t69wd\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") " pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.492842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.492793    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:25.620661 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:25.620633    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbf9947f5-t69wd"]
Jul 24 22:55:25.623397 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:55:25.623371    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda34048_c829_4281_a3a1_178c440038a9.slice/crio-747bdaa334840689775c8265f7dc6d95d23a4a4f857a6925b2ccdf5eab0311ec WatchSource:0}: Error finding container 747bdaa334840689775c8265f7dc6d95d23a4a4f857a6925b2ccdf5eab0311ec: Status 404 returned error can't find the container with id 747bdaa334840689775c8265f7dc6d95d23a4a4f857a6925b2ccdf5eab0311ec
Jul 24 22:55:26.580651 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:26.580613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf9947f5-t69wd" event={"ID":"dda34048-c829-4281-a3a1-178c440038a9","Type":"ContainerStarted","Data":"93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1"}
Jul 24 22:55:26.581050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:26.580711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf9947f5-t69wd" event={"ID":"dda34048-c829-4281-a3a1-178c440038a9","Type":"ContainerStarted","Data":"747bdaa334840689775c8265f7dc6d95d23a4a4f857a6925b2ccdf5eab0311ec"}
Jul 24 22:55:26.600201 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:26.600156    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dbf9947f5-t69wd" podStartSLOduration=1.600118244 podStartE2EDuration="1.600118244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:26.598679681 +0000 UTC m=+187.452006221" watchObservedRunningTime="2026-07-24 22:55:26.600118244 +0000 UTC m=+187.453444784"
Jul 24 22:55:35.493160 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.493108    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:35.493614 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.493189    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:35.497942 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.497919    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:35.618429 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.618402    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:55:35.662224 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.662194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b5c45d-lcgvg"]
Jul 24 22:55:35.767581 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.767501    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:35.818600 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.818561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:35.904102 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:35.904069    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:41.146611 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:41.146572    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:41.197897 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:41.197863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:56.670620 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:56.670586    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:56.774384 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:56.774352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:55:57.328174 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.328135    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"]
Jul 24 22:55:57.330605 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.330589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.333019 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.332992    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zp8l7\""
Jul 24 22:55:57.333159 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.333040    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 22:55:57.333792 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.333778    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 22:55:57.338799 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.338780    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"]
Jul 24 22:55:57.428952 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.428896    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvwln\" (UniqueName: \"kubernetes.io/projected/6514e0ea-0324-46ed-959d-c0f6436e9668-kube-api-access-mvwln\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.428952 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.428948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.429175 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.428975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.529389 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.529363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvwln\" (UniqueName: \"kubernetes.io/projected/6514e0ea-0324-46ed-959d-c0f6436e9668-kube-api-access-mvwln\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.529480 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.529397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.529480 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.529420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.529837 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.529816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.529915 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.529831    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.537868 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.537834    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvwln\" (UniqueName: \"kubernetes.io/projected/6514e0ea-0324-46ed-959d-c0f6436e9668-kube-api-access-mvwln\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.640279 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.640260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:55:57.770304 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:57.770100    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"]
Jul 24 22:55:58.702366 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:55:58.702332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd" event={"ID":"6514e0ea-0324-46ed-959d-c0f6436e9668","Type":"ContainerStarted","Data":"94af5079de34ddd502b9d8a1044a7daa12332e144f3507c40959897c1455aa4e"}
Jul 24 22:56:00.682158 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:00.682091    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b5c45d-lcgvg" podUID="f4ed7f03-09a3-4752-b963-fe1add95dadc" containerName="console" containerID="cri-o://aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219" gracePeriod=15
Jul 24 22:56:00.940776 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:00.940724    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:56:01.060161 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060118    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-trusted-ca-bundle\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060339 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060183    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-oauth-config\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060339 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060219    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-oauth-serving-cert\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060339 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060317    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-config\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060492 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060350    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-serving-cert\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060492 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-service-ca\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060492 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060457    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wlgt\" (UniqueName: \"kubernetes.io/projected/f4ed7f03-09a3-4752-b963-fe1add95dadc-kube-api-access-5wlgt\") pod \"f4ed7f03-09a3-4752-b963-fe1add95dadc\" (UID: \"f4ed7f03-09a3-4752-b963-fe1add95dadc\") "
Jul 24 22:56:01.060645 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060601    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:56:01.060925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060715    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-trusted-ca-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.060925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060848    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-config" (OuterVolumeSpecName: "console-config") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:56:01.060925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060859    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:56:01.061147 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.060903    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-service-ca" (OuterVolumeSpecName: "service-ca") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:56:01.062906 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.062855    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:56:01.062906 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.062867    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ed7f03-09a3-4752-b963-fe1add95dadc-kube-api-access-5wlgt" (OuterVolumeSpecName: "kube-api-access-5wlgt") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "kube-api-access-5wlgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:56:01.063066 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.062936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f4ed7f03-09a3-4752-b963-fe1add95dadc" (UID: "f4ed7f03-09a3-4752-b963-fe1add95dadc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:56:01.161869 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.161836    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wlgt\" (UniqueName: \"kubernetes.io/projected/f4ed7f03-09a3-4752-b963-fe1add95dadc-kube-api-access-5wlgt\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.161869 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.161870    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-oauth-config\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.162046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.161884    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-oauth-serving-cert\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.162046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.161898    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-config\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.162046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.161914    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4ed7f03-09a3-4752-b963-fe1add95dadc-console-serving-cert\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.162046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.161929    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ed7f03-09a3-4752-b963-fe1add95dadc-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:01.715267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.715232    2564 generic.go:356] "Generic (PLEG): container finished" podID="f4ed7f03-09a3-4752-b963-fe1add95dadc" containerID="aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219" exitCode=0
Jul 24 22:56:01.715557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.715281    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5c45d-lcgvg"
Jul 24 22:56:01.715557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.715311    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5c45d-lcgvg" event={"ID":"f4ed7f03-09a3-4752-b963-fe1add95dadc","Type":"ContainerDied","Data":"aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219"}
Jul 24 22:56:01.715557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.715357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5c45d-lcgvg" event={"ID":"f4ed7f03-09a3-4752-b963-fe1add95dadc","Type":"ContainerDied","Data":"760aa756cbf3200cfbf1b5ccd6ebe8e58326aa2e78a4abb8cc9d4084a5913130"}
Jul 24 22:56:01.715557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.715377    2564 scope.go:117] "RemoveContainer" containerID="aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219"
Jul 24 22:56:01.724568 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.724553    2564 scope.go:117] "RemoveContainer" containerID="aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219"
Jul 24 22:56:01.724827 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:56:01.724810    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219\": container with ID starting with aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219 not found: ID does not exist" containerID="aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219"
Jul 24 22:56:01.724935 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.724834    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219"} err="failed to get container status \"aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219\": rpc error: code = NotFound desc = could not find container \"aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219\": container with ID starting with aaf282d5f9f0d6e900b06ff35adb1382c3415c6f8d254de45bdfdc087cb0d219 not found: ID does not exist"
Jul 24 22:56:01.733254 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.733229    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b5c45d-lcgvg"]
Jul 24 22:56:01.734845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:01.734826    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b5c45d-lcgvg"]
Jul 24 22:56:02.137557 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:02.137520    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:02.190067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:02.190035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:03.698168 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:03.698108    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ed7f03-09a3-4752-b963-fe1add95dadc" path="/var/lib/kubelet/pods/f4ed7f03-09a3-4752-b963-fe1add95dadc/volumes"
Jul 24 22:56:04.731067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:04.731033    2564 generic.go:356] "Generic (PLEG): container finished" podID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerID="41c327e334827a77dfbf19fac4b7fa8950849f0cb7ed55287878aa911c207e1f" exitCode=0
Jul 24 22:56:04.731515 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:04.731137    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd" event={"ID":"6514e0ea-0324-46ed-959d-c0f6436e9668","Type":"ContainerDied","Data":"41c327e334827a77dfbf19fac4b7fa8950849f0cb7ed55287878aa911c207e1f"}
Jul 24 22:56:07.744591 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:07.744560    2564 generic.go:356] "Generic (PLEG): container finished" podID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerID="f119bd12a2e54f16993f3bc96e1835b1ffcf9c50dafebe126b3b379f24210bfd" exitCode=0
Jul 24 22:56:07.745041 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:07.744602    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd" event={"ID":"6514e0ea-0324-46ed-959d-c0f6436e9668","Type":"ContainerDied","Data":"f119bd12a2e54f16993f3bc96e1835b1ffcf9c50dafebe126b3b379f24210bfd"}
Jul 24 22:56:15.777659 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:15.777624    2564 generic.go:356] "Generic (PLEG): container finished" podID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerID="6269f6501361536088ed51754a65c2cd1467c12c3b2cf03722821da06935a1f6" exitCode=0
Jul 24 22:56:15.778051 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:15.777679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd" event={"ID":"6514e0ea-0324-46ed-959d-c0f6436e9668","Type":"ContainerDied","Data":"6269f6501361536088ed51754a65c2cd1467c12c3b2cf03722821da06935a1f6"}
Jul 24 22:56:16.909455 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:16.909430    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:56:16.998191 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:16.998160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvwln\" (UniqueName: \"kubernetes.io/projected/6514e0ea-0324-46ed-959d-c0f6436e9668-kube-api-access-mvwln\") pod \"6514e0ea-0324-46ed-959d-c0f6436e9668\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") "
Jul 24 22:56:16.998365 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:16.998231    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-util\") pod \"6514e0ea-0324-46ed-959d-c0f6436e9668\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") "
Jul 24 22:56:16.998365 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:16.998266    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-bundle\") pod \"6514e0ea-0324-46ed-959d-c0f6436e9668\" (UID: \"6514e0ea-0324-46ed-959d-c0f6436e9668\") "
Jul 24 22:56:16.999011 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:16.998973    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-bundle" (OuterVolumeSpecName: "bundle") pod "6514e0ea-0324-46ed-959d-c0f6436e9668" (UID: "6514e0ea-0324-46ed-959d-c0f6436e9668"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:56:17.000456 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.000426    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6514e0ea-0324-46ed-959d-c0f6436e9668-kube-api-access-mvwln" (OuterVolumeSpecName: "kube-api-access-mvwln") pod "6514e0ea-0324-46ed-959d-c0f6436e9668" (UID: "6514e0ea-0324-46ed-959d-c0f6436e9668"). InnerVolumeSpecName "kube-api-access-mvwln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:56:17.002837 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.002813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-util" (OuterVolumeSpecName: "util") pod "6514e0ea-0324-46ed-959d-c0f6436e9668" (UID: "6514e0ea-0324-46ed-959d-c0f6436e9668"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:56:17.099517 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.099492    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:17.099635 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.099540    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6514e0ea-0324-46ed-959d-c0f6436e9668-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:17.099635 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.099555    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvwln\" (UniqueName: \"kubernetes.io/projected/6514e0ea-0324-46ed-959d-c0f6436e9668-kube-api-access-mvwln\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:17.786749 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.786706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd" event={"ID":"6514e0ea-0324-46ed-959d-c0f6436e9668","Type":"ContainerDied","Data":"94af5079de34ddd502b9d8a1044a7daa12332e144f3507c40959897c1455aa4e"}
Jul 24 22:56:17.786749 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.786747    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cqcwd"
Jul 24 22:56:17.786922 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:17.786762    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94af5079de34ddd502b9d8a1044a7daa12332e144f3507c40959897c1455aa4e"
Jul 24 22:56:23.151671 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:23.151637    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:24.503898 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:24.503864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:25.103624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103594    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"]
Jul 24 22:56:25.103881 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103870    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="util"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103882    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="util"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103889    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="pull"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103894    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="pull"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103912    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="extract"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103917    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="extract"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103924    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ed7f03-09a3-4752-b963-fe1add95dadc" containerName="console"
Jul 24 22:56:25.103932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103929    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ed7f03-09a3-4752-b963-fe1add95dadc" containerName="console"
Jul 24 22:56:25.104139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103983    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6514e0ea-0324-46ed-959d-c0f6436e9668" containerName="extract"
Jul 24 22:56:25.104139 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.103991    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ed7f03-09a3-4752-b963-fe1add95dadc" containerName="console"
Jul 24 22:56:25.108262 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.108244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.110850 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.110831    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:56:25.111065 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.111051    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7zzww\""
Jul 24 22:56:25.111163 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.111097    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 24 22:56:25.120192 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.120167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"]
Jul 24 22:56:25.158421 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.158394    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c173f9f-f619-48ac-a26a-fe6f16425c9f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-klkmz\" (UID: \"8c173f9f-f619-48ac-a26a-fe6f16425c9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.158568 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.158492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpvr\" (UniqueName: \"kubernetes.io/projected/8c173f9f-f619-48ac-a26a-fe6f16425c9f-kube-api-access-5cpvr\") pod \"cert-manager-operator-controller-manager-55c444bf86-klkmz\" (UID: \"8c173f9f-f619-48ac-a26a-fe6f16425c9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.259288 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.259256    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c173f9f-f619-48ac-a26a-fe6f16425c9f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-klkmz\" (UID: \"8c173f9f-f619-48ac-a26a-fe6f16425c9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.259449 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.259300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpvr\" (UniqueName: \"kubernetes.io/projected/8c173f9f-f619-48ac-a26a-fe6f16425c9f-kube-api-access-5cpvr\") pod \"cert-manager-operator-controller-manager-55c444bf86-klkmz\" (UID: \"8c173f9f-f619-48ac-a26a-fe6f16425c9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.259665 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.259646    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c173f9f-f619-48ac-a26a-fe6f16425c9f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-klkmz\" (UID: \"8c173f9f-f619-48ac-a26a-fe6f16425c9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.268632 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.268610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpvr\" (UniqueName: \"kubernetes.io/projected/8c173f9f-f619-48ac-a26a-fe6f16425c9f-kube-api-access-5cpvr\") pod \"cert-manager-operator-controller-manager-55c444bf86-klkmz\" (UID: \"8c173f9f-f619-48ac-a26a-fe6f16425c9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.417336 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.417260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"
Jul 24 22:56:25.556230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.556204    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz"]
Jul 24 22:56:25.559961 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:56:25.559938    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c173f9f_f619_48ac_a26a_fe6f16425c9f.slice/crio-bb92a81a7bd2f207aae628a28f18ef9737f61aaf480635555063e22448e3b08a WatchSource:0}: Error finding container bb92a81a7bd2f207aae628a28f18ef9737f61aaf480635555063e22448e3b08a: Status 404 returned error can't find the container with id bb92a81a7bd2f207aae628a28f18ef9737f61aaf480635555063e22448e3b08a
Jul 24 22:56:25.817736 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:25.817652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz" event={"ID":"8c173f9f-f619-48ac-a26a-fe6f16425c9f","Type":"ContainerStarted","Data":"bb92a81a7bd2f207aae628a28f18ef9737f61aaf480635555063e22448e3b08a"}
Jul 24 22:56:26.770183 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:26.770143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:26.873449 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:26.873394    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:27.826889 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:27.826820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz" event={"ID":"8c173f9f-f619-48ac-a26a-fe6f16425c9f","Type":"ContainerStarted","Data":"9f141aa2e70a2d9fee995cd5c98361390cb590f616a493f48d9d6d6114855736"}
Jul 24 22:56:27.846532 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:27.846489    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-klkmz" podStartSLOduration=0.876486652 podStartE2EDuration="2.846473116s" totalImagesPullingTime="1.969986464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:56:27.845425394 +0000 UTC m=+248.698751935" watchObservedRunningTime="2026-07-24 22:56:27.846473116 +0000 UTC m=+248.699799655"
Jul 24 22:56:28.300675 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:28.300638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:28.421381 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:28.421336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:29.778818 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.778782    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"]
Jul 24 22:56:29.782421 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.782400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.784722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.784702    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 22:56:29.784816 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.784737    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 22:56:29.785464 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.785445    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zp8l7\""
Jul 24 22:56:29.790049 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.790015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"]
Jul 24 22:56:29.898382 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.898350    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwz2x\" (UniqueName: \"kubernetes.io/projected/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-kube-api-access-fwz2x\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.898537 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.898401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.898537 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.898471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.999440 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.999415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.999577 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.999452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.999577 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.999490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwz2x\" (UniqueName: \"kubernetes.io/projected/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-kube-api-access-fwz2x\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.999845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.999821    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:29.999882 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:29.999839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:30.008633 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.008611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwz2x\" (UniqueName: \"kubernetes.io/projected/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-kube-api-access-fwz2x\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:30.091868 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.091850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:30.218190 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.218159    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"]
Jul 24 22:56:30.220839 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:56:30.220806    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e03dde6_5bd1_4d0c_8708_6a70f0095df9.slice/crio-56252e76cda244f662241f5440feb5bbbb675f87a686230037138d6f320c2303 WatchSource:0}: Error finding container 56252e76cda244f662241f5440feb5bbbb675f87a686230037138d6f320c2303: Status 404 returned error can't find the container with id 56252e76cda244f662241f5440feb5bbbb675f87a686230037138d6f320c2303
Jul 24 22:56:30.639973 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.639944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:30.831764 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.831728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:30.839225 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.839196    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerID="96a298a121e1efcc1fcab97fb99b106321c7b375bc7a487f6411e4034a02a84c" exitCode=0
Jul 24 22:56:30.839338 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.839279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77" event={"ID":"1e03dde6-5bd1-4d0c-8708-6a70f0095df9","Type":"ContainerDied","Data":"96a298a121e1efcc1fcab97fb99b106321c7b375bc7a487f6411e4034a02a84c"}
Jul 24 22:56:30.839376 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:30.839347    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77" event={"ID":"1e03dde6-5bd1-4d0c-8708-6a70f0095df9","Type":"ContainerStarted","Data":"56252e76cda244f662241f5440feb5bbbb675f87a686230037138d6f320c2303"}
Jul 24 22:56:31.019879 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.019855    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"]
Jul 24 22:56:31.022988 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.022974    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.025338 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.025317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 22:56:31.025427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.025338    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-92rnw\""
Jul 24 22:56:31.025427 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.025322    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 22:56:31.033308 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.033285    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"]
Jul 24 22:56:31.109212 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.109180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f60d2124-6a94-43d4-9128-ed5762074b58-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7vjv6\" (UID: \"f60d2124-6a94-43d4-9128-ed5762074b58\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.109352 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.109226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dpd4\" (UniqueName: \"kubernetes.io/projected/f60d2124-6a94-43d4-9128-ed5762074b58-kube-api-access-4dpd4\") pod \"cert-manager-cainjector-559cdd8c87-7vjv6\" (UID: \"f60d2124-6a94-43d4-9128-ed5762074b58\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.210470 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.210438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f60d2124-6a94-43d4-9128-ed5762074b58-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7vjv6\" (UID: \"f60d2124-6a94-43d4-9128-ed5762074b58\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.210612 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.210477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dpd4\" (UniqueName: \"kubernetes.io/projected/f60d2124-6a94-43d4-9128-ed5762074b58-kube-api-access-4dpd4\") pod \"cert-manager-cainjector-559cdd8c87-7vjv6\" (UID: \"f60d2124-6a94-43d4-9128-ed5762074b58\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.218892 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.218859    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f60d2124-6a94-43d4-9128-ed5762074b58-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7vjv6\" (UID: \"f60d2124-6a94-43d4-9128-ed5762074b58\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.219085 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.219066    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dpd4\" (UniqueName: \"kubernetes.io/projected/f60d2124-6a94-43d4-9128-ed5762074b58-kube-api-access-4dpd4\") pod \"cert-manager-cainjector-559cdd8c87-7vjv6\" (UID: \"f60d2124-6a94-43d4-9128-ed5762074b58\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.332138 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.332103    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"
Jul 24 22:56:31.462705 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.462684    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6"]
Jul 24 22:56:31.464464 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:56:31.464441    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf60d2124_6a94_43d4_9128_ed5762074b58.slice/crio-a46fac5ac76b77142437fb507f9c4ea60a22544b59c5be21d7f05eef0614a6d6 WatchSource:0}: Error finding container a46fac5ac76b77142437fb507f9c4ea60a22544b59c5be21d7f05eef0614a6d6: Status 404 returned error can't find the container with id a46fac5ac76b77142437fb507f9c4ea60a22544b59c5be21d7f05eef0614a6d6
Jul 24 22:56:31.845138 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:31.845084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6" event={"ID":"f60d2124-6a94-43d4-9128-ed5762074b58","Type":"ContainerStarted","Data":"a46fac5ac76b77142437fb507f9c4ea60a22544b59c5be21d7f05eef0614a6d6"}
Jul 24 22:56:33.557831 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:33.557794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:33.658552 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:33.658504    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:33.855405 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:33.855372    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerID="6017c4f805b4ffbb858e4e0c4262317f52904ecc2931cea78b404435b320a61c" exitCode=0
Jul 24 22:56:33.855573 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:33.855447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77" event={"ID":"1e03dde6-5bd1-4d0c-8708-6a70f0095df9","Type":"ContainerDied","Data":"6017c4f805b4ffbb858e4e0c4262317f52904ecc2931cea78b404435b320a61c"}
Jul 24 22:56:34.863532 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:34.863496    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerID="853a9ab6029419262867168cce801174bc2711ccf145e916037cd2c7b74edb51" exitCode=0
Jul 24 22:56:34.863957 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:34.863578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77" event={"ID":"1e03dde6-5bd1-4d0c-8708-6a70f0095df9","Type":"ContainerDied","Data":"853a9ab6029419262867168cce801174bc2711ccf145e916037cd2c7b74edb51"}
Jul 24 22:56:35.869486 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:35.869458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6" event={"ID":"f60d2124-6a94-43d4-9128-ed5762074b58","Type":"ContainerStarted","Data":"ddeb5bdc943dbb722d290c4aeb9ecf01e40ae11630dc42253b166684a751ed8b"}
Jul 24 22:56:35.903785 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:35.903733    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7vjv6" podStartSLOduration=1.083142096 podStartE2EDuration="4.90371608s" totalImagesPullingTime="3.820573984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:56:35.902464105 +0000 UTC m=+256.755790645" watchObservedRunningTime="2026-07-24 22:56:35.90371608 +0000 UTC m=+256.757042617"
Jul 24 22:56:36.010449 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.010428    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:36.153354 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.153292    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-bundle\") pod \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") "
Jul 24 22:56:36.153354 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.153345    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwz2x\" (UniqueName: \"kubernetes.io/projected/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-kube-api-access-fwz2x\") pod \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") "
Jul 24 22:56:36.153503 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.153375    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-util\") pod \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\" (UID: \"1e03dde6-5bd1-4d0c-8708-6a70f0095df9\") "
Jul 24 22:56:36.153695 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.153672    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-bundle" (OuterVolumeSpecName: "bundle") pod "1e03dde6-5bd1-4d0c-8708-6a70f0095df9" (UID: "1e03dde6-5bd1-4d0c-8708-6a70f0095df9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:56:36.155553 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.155528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-kube-api-access-fwz2x" (OuterVolumeSpecName: "kube-api-access-fwz2x") pod "1e03dde6-5bd1-4d0c-8708-6a70f0095df9" (UID: "1e03dde6-5bd1-4d0c-8708-6a70f0095df9"). InnerVolumeSpecName "kube-api-access-fwz2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:56:36.158017 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.157995    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-util" (OuterVolumeSpecName: "util") pod "1e03dde6-5bd1-4d0c-8708-6a70f0095df9" (UID: "1e03dde6-5bd1-4d0c-8708-6a70f0095df9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:56:36.254522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.254494    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwz2x\" (UniqueName: \"kubernetes.io/projected/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-kube-api-access-fwz2x\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:36.254522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.254517    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:36.254659 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.254532    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e03dde6-5bd1-4d0c-8708-6a70f0095df9-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.594721    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z7zcf"]
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595423    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="util"
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595441    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="util"
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595483    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="extract"
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595491    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="extract"
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595508    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="pull"
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595517    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="pull"
Jul 24 22:56:36.597023 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.595658    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e03dde6-5bd1-4d0c-8708-6a70f0095df9" containerName="extract"
Jul 24 22:56:36.598453 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.598422    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.601104 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.601083    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-clz7c\""
Jul 24 22:56:36.604454 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.604434    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z7zcf"]
Jul 24 22:56:36.657478 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.657457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jx69\" (UniqueName: \"kubernetes.io/projected/93b52b76-8819-4a4f-ab55-40d5f4ed554c-kube-api-access-2jx69\") pod \"cert-manager-webhook-6486ffd997-z7zcf\" (UID: \"93b52b76-8819-4a4f-ab55-40d5f4ed554c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.657566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.657486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93b52b76-8819-4a4f-ab55-40d5f4ed554c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z7zcf\" (UID: \"93b52b76-8819-4a4f-ab55-40d5f4ed554c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.757842 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.757815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jx69\" (UniqueName: \"kubernetes.io/projected/93b52b76-8819-4a4f-ab55-40d5f4ed554c-kube-api-access-2jx69\") pod \"cert-manager-webhook-6486ffd997-z7zcf\" (UID: \"93b52b76-8819-4a4f-ab55-40d5f4ed554c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.757981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.757849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93b52b76-8819-4a4f-ab55-40d5f4ed554c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z7zcf\" (UID: \"93b52b76-8819-4a4f-ab55-40d5f4ed554c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.766754 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.766728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93b52b76-8819-4a4f-ab55-40d5f4ed554c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z7zcf\" (UID: \"93b52b76-8819-4a4f-ab55-40d5f4ed554c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.766882 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.766865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jx69\" (UniqueName: \"kubernetes.io/projected/93b52b76-8819-4a4f-ab55-40d5f4ed554c-kube-api-access-2jx69\") pod \"cert-manager-webhook-6486ffd997-z7zcf\" (UID: \"93b52b76-8819-4a4f-ab55-40d5f4ed554c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:36.875850 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.875789    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77"
Jul 24 22:56:36.875850 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.875810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fslg77" event={"ID":"1e03dde6-5bd1-4d0c-8708-6a70f0095df9","Type":"ContainerDied","Data":"56252e76cda244f662241f5440feb5bbbb675f87a686230037138d6f320c2303"}
Jul 24 22:56:36.875850 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.875850    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56252e76cda244f662241f5440feb5bbbb675f87a686230037138d6f320c2303"
Jul 24 22:56:36.908920 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:36.908899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:37.035627 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.035606    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z7zcf"]
Jul 24 22:56:37.038076 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:56:37.038047    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93b52b76_8819_4a4f_ab55_40d5f4ed554c.slice/crio-b10219535d0b39eb578f39bace733f1e21350c4f3e42019c47218832cca6c47f WatchSource:0}: Error finding container b10219535d0b39eb578f39bace733f1e21350c4f3e42019c47218832cca6c47f: Status 404 returned error can't find the container with id b10219535d0b39eb578f39bace733f1e21350c4f3e42019c47218832cca6c47f
Jul 24 22:56:37.299929 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.299849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:37.402147 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.402080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:37.880805 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.880775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf" event={"ID":"93b52b76-8819-4a4f-ab55-40d5f4ed554c","Type":"ContainerStarted","Data":"49efb6e83d1366e17c3b93b2cfcede9785640db92d2e22b26839214d6d108f8a"}
Jul 24 22:56:37.881207 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.880817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf" event={"ID":"93b52b76-8819-4a4f-ab55-40d5f4ed554c","Type":"ContainerStarted","Data":"b10219535d0b39eb578f39bace733f1e21350c4f3e42019c47218832cca6c47f"}
Jul 24 22:56:37.881207 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.880866    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:37.898920 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:37.898872    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf" podStartSLOduration=1.898858581 podStartE2EDuration="1.898858581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:56:37.897460857 +0000 UTC m=+258.750787396" watchObservedRunningTime="2026-07-24 22:56:37.898858581 +0000 UTC m=+258.752185119"
Jul 24 22:56:38.235531 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:38.235458    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:39.675168 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.675135    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lbwh9"]
Jul 24 22:56:39.677437 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.677422    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.679553 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.679528    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-twqvq\""
Jul 24 22:56:39.686428 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.686408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lbwh9"]
Jul 24 22:56:39.781978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.781956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac4f2746-c1d4-4035-8986-b3a0d4a23527-bound-sa-token\") pod \"cert-manager-55b4d954d5-lbwh9\" (UID: \"ac4f2746-c1d4-4035-8986-b3a0d4a23527\") " pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.782148 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.781995    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pxzl\" (UniqueName: \"kubernetes.io/projected/ac4f2746-c1d4-4035-8986-b3a0d4a23527-kube-api-access-7pxzl\") pod \"cert-manager-55b4d954d5-lbwh9\" (UID: \"ac4f2746-c1d4-4035-8986-b3a0d4a23527\") " pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.882687 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.882663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac4f2746-c1d4-4035-8986-b3a0d4a23527-bound-sa-token\") pod \"cert-manager-55b4d954d5-lbwh9\" (UID: \"ac4f2746-c1d4-4035-8986-b3a0d4a23527\") " pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.882766 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.882701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pxzl\" (UniqueName: \"kubernetes.io/projected/ac4f2746-c1d4-4035-8986-b3a0d4a23527-kube-api-access-7pxzl\") pod \"cert-manager-55b4d954d5-lbwh9\" (UID: \"ac4f2746-c1d4-4035-8986-b3a0d4a23527\") " pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.890569 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.890540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac4f2746-c1d4-4035-8986-b3a0d4a23527-bound-sa-token\") pod \"cert-manager-55b4d954d5-lbwh9\" (UID: \"ac4f2746-c1d4-4035-8986-b3a0d4a23527\") " pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.891098 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.891076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pxzl\" (UniqueName: \"kubernetes.io/projected/ac4f2746-c1d4-4035-8986-b3a0d4a23527-kube-api-access-7pxzl\") pod \"cert-manager-55b4d954d5-lbwh9\" (UID: \"ac4f2746-c1d4-4035-8986-b3a0d4a23527\") " pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:39.987800 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:39.987744    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lbwh9"
Jul 24 22:56:40.111227 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:40.111203    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lbwh9"]
Jul 24 22:56:40.113322 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:56:40.113291    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac4f2746_c1d4_4035_8986_b3a0d4a23527.slice/crio-b069b4b2d12dd8c914008292b4fef8f347816adedfef7dff179fdd3257d9fd86 WatchSource:0}: Error finding container b069b4b2d12dd8c914008292b4fef8f347816adedfef7dff179fdd3257d9fd86: Status 404 returned error can't find the container with id b069b4b2d12dd8c914008292b4fef8f347816adedfef7dff179fdd3257d9fd86
Jul 24 22:56:40.895767 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:40.895734    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lbwh9" event={"ID":"ac4f2746-c1d4-4035-8986-b3a0d4a23527","Type":"ContainerStarted","Data":"a1622a4031d99e7b28915edbb1fc9a6e3006a7b4af71c27b930fee1ece8f570a"}
Jul 24 22:56:40.896112 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:40.895779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lbwh9" event={"ID":"ac4f2746-c1d4-4035-8986-b3a0d4a23527","Type":"ContainerStarted","Data":"b069b4b2d12dd8c914008292b4fef8f347816adedfef7dff179fdd3257d9fd86"}
Jul 24 22:56:40.912560 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:40.912514    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lbwh9" podStartSLOduration=1.912500596 podStartE2EDuration="1.912500596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:56:40.911070188 +0000 UTC m=+261.764396725" watchObservedRunningTime="2026-07-24 22:56:40.912500596 +0000 UTC m=+261.765827135"
Jul 24 22:56:41.432716 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:41.432677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:41.528309 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:41.528276    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:43.887699 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:43.887673    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z7zcf"
Jul 24 22:56:47.181443 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:47.181400    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:47.280808 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:47.280777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:49.047895 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.047856    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"]
Jul 24 22:56:49.078714 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.078682    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"]
Jul 24 22:56:49.078877 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.078770    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.081629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.081607    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 22:56:49.081735 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.081611    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 22:56:49.082242 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.082229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zp8l7\""
Jul 24 22:56:49.153876 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.153846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.154006 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.153883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.154006 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.153937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2px7j\" (UniqueName: \"kubernetes.io/projected/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-kube-api-access-2px7j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.172579 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.172539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:56:49.255333 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.255311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.255431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.255337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.255431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.255373    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2px7j\" (UniqueName: \"kubernetes.io/projected/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-kube-api-access-2px7j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.255738 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.255715    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.255774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.255728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.263307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.263277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2px7j\" (UniqueName: \"kubernetes.io/projected/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-kube-api-access-2px7j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.388367 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.388343    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:49.525858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.525834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"]
Jul 24 22:56:49.527931 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:56:49.527890    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bbdff0_192c_4a05_9c8b_f183d3c04e5b.slice/crio-abb803a8dc96f419cf08e37c69107b654f658ea5fd91727e502bb2cf5487c180 WatchSource:0}: Error finding container abb803a8dc96f419cf08e37c69107b654f658ea5fd91727e502bb2cf5487c180: Status 404 returned error can't find the container with id abb803a8dc96f419cf08e37c69107b654f658ea5fd91727e502bb2cf5487c180
Jul 24 22:56:49.935654 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.935568    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerID="eb568f5960eb125d700b6c30c9638585c0514804f0a9c4f03c90f3f0842c4102" exitCode=0
Jul 24 22:56:49.935654 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.935620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4" event={"ID":"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b","Type":"ContainerDied","Data":"eb568f5960eb125d700b6c30c9638585c0514804f0a9c4f03c90f3f0842c4102"}
Jul 24 22:56:49.935869 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:49.935665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4" event={"ID":"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b","Type":"ContainerStarted","Data":"abb803a8dc96f419cf08e37c69107b654f658ea5fd91727e502bb2cf5487c180"}
Jul 24 22:56:50.941920 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:50.941887    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerID="a4364e082b6cff45537693e900075b6ceb80ea76922ba0d3701408060252a8cf" exitCode=0
Jul 24 22:56:50.942403 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:50.941934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4" event={"ID":"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b","Type":"ContainerDied","Data":"a4364e082b6cff45537693e900075b6ceb80ea76922ba0d3701408060252a8cf"}
Jul 24 22:56:51.948727 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:51.948689    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerID="bf92a41deb9a3bef5f0fb74e3624cf477f4c194f39959bf0d65f13021bc400df" exitCode=0
Jul 24 22:56:51.949137 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:51.948736    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4" event={"ID":"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b","Type":"ContainerDied","Data":"bf92a41deb9a3bef5f0fb74e3624cf477f4c194f39959bf0d65f13021bc400df"}
Jul 24 22:56:53.078295 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.078273    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:56:53.188395 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.188366    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-util\") pod \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") "
Jul 24 22:56:53.188395 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.188396    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2px7j\" (UniqueName: \"kubernetes.io/projected/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-kube-api-access-2px7j\") pod \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") "
Jul 24 22:56:53.188572 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.188481    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-bundle\") pod \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\" (UID: \"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b\") "
Jul 24 22:56:53.189237 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.189206    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-bundle" (OuterVolumeSpecName: "bundle") pod "f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" (UID: "f7bbdff0-192c-4a05-9c8b-f183d3c04e5b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:56:53.190469 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.190442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-kube-api-access-2px7j" (OuterVolumeSpecName: "kube-api-access-2px7j") pod "f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" (UID: "f7bbdff0-192c-4a05-9c8b-f183d3c04e5b"). InnerVolumeSpecName "kube-api-access-2px7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:56:53.193828 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.193785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-util" (OuterVolumeSpecName: "util") pod "f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" (UID: "f7bbdff0-192c-4a05-9c8b-f183d3c04e5b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:56:53.290058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.289995    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:53.290058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.290020    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2px7j\" (UniqueName: \"kubernetes.io/projected/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-kube-api-access-2px7j\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:53.290058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.290031    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7bbdff0-192c-4a05-9c8b-f183d3c04e5b-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:56:53.959071 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.958995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4" event={"ID":"f7bbdff0-192c-4a05-9c8b-f183d3c04e5b","Type":"ContainerDied","Data":"abb803a8dc96f419cf08e37c69107b654f658ea5fd91727e502bb2cf5487c180"}
Jul 24 22:56:53.959071 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.959037    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abb803a8dc96f419cf08e37c69107b654f658ea5fd91727e502bb2cf5487c180"
Jul 24 22:56:53.959071 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:56:53.959047    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5qnn4"
Jul 24 22:57:02.816898 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:02.816858    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:02.926944 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:02.926906    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:03.401471 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.401437    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:03.615925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.615898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:03.726317 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.726241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:03.775584 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775543    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"]
Jul 24 22:57:03.775836 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775825    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="extract"
Jul 24 22:57:03.775836 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775837    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="extract"
Jul 24 22:57:03.775905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775852    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="util"
Jul 24 22:57:03.775905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775858    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="util"
Jul 24 22:57:03.775905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775872    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="pull"
Jul 24 22:57:03.775905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775877    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="pull"
Jul 24 22:57:03.776022 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.775924    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7bbdff0-192c-4a05-9c8b-f183d3c04e5b" containerName="extract"
Jul 24 22:57:03.781902 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.781881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:03.784720 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.784703    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 22:57:03.784905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.784877    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 22:57:03.785032 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.785015    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 22:57:03.785098 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.785016    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7r678\""
Jul 24 22:57:03.785192 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.785101    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 22:57:03.794903 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.794876    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"]
Jul 24 22:57:03.921603 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.921576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:03.974732 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.974706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljdqn\" (UniqueName: \"kubernetes.io/projected/f26aa5f2-483b-449d-a43a-c018b9c75ebc-kube-api-access-ljdqn\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:03.974849 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.974738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f26aa5f2-483b-449d-a43a-c018b9c75ebc-webhook-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:03.974849 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.974761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f26aa5f2-483b-449d-a43a-c018b9c75ebc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:03.974849 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.974810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f26aa5f2-483b-449d-a43a-c018b9c75ebc-manifests\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:03.974942 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:03.974872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f26aa5f2-483b-449d-a43a-c018b9c75ebc-tmp\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.022936 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.022874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:04.075213 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.075190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljdqn\" (UniqueName: \"kubernetes.io/projected/f26aa5f2-483b-449d-a43a-c018b9c75ebc-kube-api-access-ljdqn\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.075296 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.075222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f26aa5f2-483b-449d-a43a-c018b9c75ebc-webhook-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.075296 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.075247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f26aa5f2-483b-449d-a43a-c018b9c75ebc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.075380 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.075363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f26aa5f2-483b-449d-a43a-c018b9c75ebc-manifests\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.075452 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.075438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f26aa5f2-483b-449d-a43a-c018b9c75ebc-tmp\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.075713 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.075691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f26aa5f2-483b-449d-a43a-c018b9c75ebc-manifests\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.077549 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.077523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f26aa5f2-483b-449d-a43a-c018b9c75ebc-tmp\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.077630 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.077601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f26aa5f2-483b-449d-a43a-c018b9c75ebc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.077671 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.077633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f26aa5f2-483b-449d-a43a-c018b9c75ebc-webhook-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.091135 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.091095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljdqn\" (UniqueName: \"kubernetes.io/projected/f26aa5f2-483b-449d-a43a-c018b9c75ebc-kube-api-access-ljdqn\") pod \"opendatahub-operator-controller-manager-777c88b5cf-qztx5\" (UID: \"f26aa5f2-483b-449d-a43a-c018b9c75ebc\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.091780 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.091766    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:04.127321 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.127295    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:04.225412 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.225384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:04.226477 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.226429    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"]
Jul 24 22:57:04.228910 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:04.228870    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26aa5f2_483b_449d_a43a_c018b9c75ebc.slice/crio-0819a038c92cbc7dcc891b3d59dfd2055b32633d0e14b1f2b2ca63deaadb3913 WatchSource:0}: Error finding container 0819a038c92cbc7dcc891b3d59dfd2055b32633d0e14b1f2b2ca63deaadb3913: Status 404 returned error can't find the container with id 0819a038c92cbc7dcc891b3d59dfd2055b32633d0e14b1f2b2ca63deaadb3913
Jul 24 22:57:04.323442 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:04.323386    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:05.017813 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:05.017779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5" event={"ID":"f26aa5f2-483b-449d-a43a-c018b9c75ebc","Type":"ContainerStarted","Data":"0819a038c92cbc7dcc891b3d59dfd2055b32633d0e14b1f2b2ca63deaadb3913"}
Jul 24 22:57:08.035945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:08.035914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5" event={"ID":"f26aa5f2-483b-449d-a43a-c018b9c75ebc","Type":"ContainerStarted","Data":"4eff80ef2281c4e9d21feda5bca13e73494662fb25c9cebb6f77053885d380b6"}
Jul 24 22:57:09.042260 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:09.042223    2564 generic.go:356] "Generic (PLEG): container finished" podID="f26aa5f2-483b-449d-a43a-c018b9c75ebc" containerID="4eff80ef2281c4e9d21feda5bca13e73494662fb25c9cebb6f77053885d380b6" exitCode=0
Jul 24 22:57:09.042643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:09.042307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5" event={"ID":"f26aa5f2-483b-449d-a43a-c018b9c75ebc","Type":"ContainerDied","Data":"4eff80ef2281c4e9d21feda5bca13e73494662fb25c9cebb6f77053885d380b6"}
Jul 24 22:57:09.401431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:09.401402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:09.502965 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:09.502929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:10.048640 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:10.048603    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5" event={"ID":"f26aa5f2-483b-449d-a43a-c018b9c75ebc","Type":"ContainerStarted","Data":"9a0008942fef5c0eefac549195b95f5b0a0f476f17ac27b21693224ebf876ec3"}
Jul 24 22:57:10.049021 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:10.048757    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:10.075547 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:10.075500    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5" podStartSLOduration=2.540263949 podStartE2EDuration="7.075487367s" totalImagesPullingTime="3.511587445s" totalInitContainerRuntime="1.023635973s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:10.074351567 +0000 UTC m=+290.927678105" watchObservedRunningTime="2026-07-24 22:57:10.075487367 +0000 UTC m=+290.928813906"
Jul 24 22:57:15.178407 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:15.178370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:15.280470 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:15.280439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:19.512563 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:19.512534    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 22:57:19.517093 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:19.517066    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 22:57:19.529918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:19.529895    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 22:57:21.056079 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:21.056048    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-qztx5"
Jul 24 22:57:22.693627 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:22.693596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:22.825002 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:22.824941    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:23.172722 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.172686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:23.192679 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.192652    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"]
Jul 24 22:57:23.196070 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.196052    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.198944 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.198927    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 22:57:23.199046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.198933    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 22:57:23.199232 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.199213    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 22:57:23.199766 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.199750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 22:57:23.212394 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.212369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"]
Jul 24 22:57:23.225348 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.225327    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22bdb04-157a-4ccd-bab8-c672428aada2-tls-certs\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.225469 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.225376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhrt\" (UniqueName: \"kubernetes.io/projected/d22bdb04-157a-4ccd-bab8-c672428aada2-kube-api-access-8dhrt\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.225469 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.225416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d22bdb04-157a-4ccd-bab8-c672428aada2-tmp\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.326566 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.326539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhrt\" (UniqueName: \"kubernetes.io/projected/d22bdb04-157a-4ccd-bab8-c672428aada2-kube-api-access-8dhrt\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.326711 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.326577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d22bdb04-157a-4ccd-bab8-c672428aada2-tmp\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.326711 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.326610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22bdb04-157a-4ccd-bab8-c672428aada2-tls-certs\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.328956 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.328913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d22bdb04-157a-4ccd-bab8-c672428aada2-tmp\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.329060 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.329028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22bdb04-157a-4ccd-bab8-c672428aada2-tls-certs\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.341083 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.341056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhrt\" (UniqueName: \"kubernetes.io/projected/d22bdb04-157a-4ccd-bab8-c672428aada2-kube-api-access-8dhrt\") pod \"kube-auth-proxy-745f8948c6-bt5q9\" (UID: \"d22bdb04-157a-4ccd-bab8-c672428aada2\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.356704 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.356663    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:23.438792 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.438733    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vthzr"]
Jul 24 22:57:23.442093 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.442077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:23.444522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.444503    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-m26t7\""
Jul 24 22:57:23.444522 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.444517    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 22:57:23.452080 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.452060    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vthzr"]
Jul 24 22:57:23.472250 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.472223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:23.506381 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.506356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"
Jul 24 22:57:23.528454 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.528419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vstw4\" (UniqueName: \"kubernetes.io/projected/a36ff974-d2d8-4de7-ac17-b7885415da66-kube-api-access-vstw4\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:23.528535 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.528494    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36ff974-d2d8-4de7-ac17-b7885415da66-cert\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:23.576665 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.576623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:23.629742 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.629716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36ff974-d2d8-4de7-ac17-b7885415da66-cert\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:23.629870 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.629773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vstw4\" (UniqueName: \"kubernetes.io/projected/a36ff974-d2d8-4de7-ac17-b7885415da66-kube-api-access-vstw4\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:23.629921 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:57:23.629867    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 24 22:57:23.629958 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:57:23.629928    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a36ff974-d2d8-4de7-ac17-b7885415da66-cert podName:a36ff974-d2d8-4de7-ac17-b7885415da66 nodeName:}" failed. No retries permitted until 2026-07-24 22:57:24.129911885 +0000 UTC m=+304.983238403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a36ff974-d2d8-4de7-ac17-b7885415da66-cert") pod "odh-model-controller-7998c67dc4-vthzr" (UID: "a36ff974-d2d8-4de7-ac17-b7885415da66") : secret "odh-model-controller-webhook-cert" not found
Jul 24 22:57:23.633353 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.633334    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9"]
Jul 24 22:57:23.635188 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:23.635166    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22bdb04_157a_4ccd_bab8_c672428aada2.slice/crio-fe609ce4cc66022bda644694e68c1a067d68923955d18c50aa0fd9c3e290ce27 WatchSource:0}: Error finding container fe609ce4cc66022bda644694e68c1a067d68923955d18c50aa0fd9c3e290ce27: Status 404 returned error can't find the container with id fe609ce4cc66022bda644694e68c1a067d68923955d18c50aa0fd9c3e290ce27
Jul 24 22:57:23.636892 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.636875    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 22:57:23.642775 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:23.642748    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vstw4\" (UniqueName: \"kubernetes.io/projected/a36ff974-d2d8-4de7-ac17-b7885415da66-kube-api-access-vstw4\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:24.012084 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.012021    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:24.114187 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.114160    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9" event={"ID":"d22bdb04-157a-4ccd-bab8-c672428aada2","Type":"ContainerStarted","Data":"fe609ce4cc66022bda644694e68c1a067d68923955d18c50aa0fd9c3e290ce27"}
Jul 24 22:57:24.134577 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.134553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36ff974-d2d8-4de7-ac17-b7885415da66-cert\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:24.136745 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.136727    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36ff974-d2d8-4de7-ac17-b7885415da66-cert\") pod \"odh-model-controller-7998c67dc4-vthzr\" (UID: \"a36ff974-d2d8-4de7-ac17-b7885415da66\") " pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:24.172116 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.172087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:24.190304 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.190278    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z79ww"]
Jul 24 22:57:24.194922 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.194904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.197199 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.197185    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 22:57:24.197421 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.197409    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6x456\""
Jul 24 22:57:24.206755 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.206734    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z79ww"]
Jul 24 22:57:24.235858 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.235837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3185626a-2701-4b83-9807-58def62c683f-tls-certs\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.235998 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.235910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6td4f\" (UniqueName: \"kubernetes.io/projected/3185626a-2701-4b83-9807-58def62c683f-kube-api-access-6td4f\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.337345 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.337306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6td4f\" (UniqueName: \"kubernetes.io/projected/3185626a-2701-4b83-9807-58def62c683f-kube-api-access-6td4f\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.337499 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.337420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3185626a-2701-4b83-9807-58def62c683f-tls-certs\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.337563 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:57:24.337528    2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 24 22:57:24.337620 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:57:24.337582    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3185626a-2701-4b83-9807-58def62c683f-tls-certs podName:3185626a-2701-4b83-9807-58def62c683f nodeName:}" failed. No retries permitted until 2026-07-24 22:57:24.837564273 +0000 UTC m=+305.690890794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3185626a-2701-4b83-9807-58def62c683f-tls-certs") pod "model-serving-api-54d89bd5fc-z79ww" (UID: "3185626a-2701-4b83-9807-58def62c683f") : secret "model-serving-api-tls" not found
Jul 24 22:57:24.353577 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.353546    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:24.358981 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.358962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6td4f\" (UniqueName: \"kubernetes.io/projected/3185626a-2701-4b83-9807-58def62c683f-kube-api-access-6td4f\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.506609 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.506584    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vthzr"]
Jul 24 22:57:24.508631 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:24.508597    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36ff974_d2d8_4de7_ac17_b7885415da66.slice/crio-bfc56b7b4f7fce99bf716fc152fc50427137c2caa5f78fa55ecee5ea5f30c85c WatchSource:0}: Error finding container bfc56b7b4f7fce99bf716fc152fc50427137c2caa5f78fa55ecee5ea5f30c85c: Status 404 returned error can't find the container with id bfc56b7b4f7fce99bf716fc152fc50427137c2caa5f78fa55ecee5ea5f30c85c
Jul 24 22:57:24.587770 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.587699    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:24.596850 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.596825    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"]
Jul 24 22:57:24.601848 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.601829    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.604705 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.604683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 22:57:24.604829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.604734    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 22:57:24.604829 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.604816    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 22:57:24.604968 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.604926    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 22:57:24.604968 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.604929    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7h7rl\""
Jul 24 22:57:24.613790 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.613772    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"]
Jul 24 22:57:24.639843 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.639818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.639945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.639873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.639945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.639903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.639945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.639930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10a7454-9520-49eb-8325-a8ece7cfc664-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.640067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.640009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.640067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.640035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.640183 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.640098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htqgp\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-kube-api-access-htqgp\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.723855 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.723825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:24.741274 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.741409 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741305    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.741484 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.741484 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10a7454-9520-49eb-8325-a8ece7cfc664-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.741581 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.741918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.741918 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.741728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqgp\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-kube-api-access-htqgp\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.742445 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.742419    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.744032 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.743990    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10a7454-9520-49eb-8325-a8ece7cfc664-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.744284 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.744264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.744370 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.744331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.744431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.744419    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.749793 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.749770    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.749895 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.749846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqgp\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-kube-api-access-htqgp\") pod \"istiod-openshift-gateway-7b7995d5cd-phrvz\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:24.842682 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.842619    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3185626a-2701-4b83-9807-58def62c683f-tls-certs\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.844775 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.844758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3185626a-2701-4b83-9807-58def62c683f-tls-certs\") pod \"model-serving-api-54d89bd5fc-z79ww\" (UID: \"3185626a-2701-4b83-9807-58def62c683f\") " pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:24.913682 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:24.913656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:25.070405 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:25.070373    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"]
Jul 24 22:57:25.071818 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:25.071791    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10a7454_9520_49eb_8325_a8ece7cfc664.slice/crio-328e17138eee4759552a263149a793a48a417cbd10ff945fc7a2d4737862e806 WatchSource:0}: Error finding container 328e17138eee4759552a263149a793a48a417cbd10ff945fc7a2d4737862e806: Status 404 returned error can't find the container with id 328e17138eee4759552a263149a793a48a417cbd10ff945fc7a2d4737862e806
Jul 24 22:57:25.106093 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:25.106040    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:25.122997 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:25.122929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" event={"ID":"b10a7454-9520-49eb-8325-a8ece7cfc664","Type":"ContainerStarted","Data":"328e17138eee4759552a263149a793a48a417cbd10ff945fc7a2d4737862e806"}
Jul 24 22:57:25.125044 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:25.124983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr" event={"ID":"a36ff974-d2d8-4de7-ac17-b7885415da66","Type":"ContainerStarted","Data":"bfc56b7b4f7fce99bf716fc152fc50427137c2caa5f78fa55ecee5ea5f30c85c"}
Jul 24 22:57:25.272294 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:25.272265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z79ww"]
Jul 24 22:57:25.275153 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:25.275098    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3185626a_2701_4b83_9807_58def62c683f.slice/crio-b8fe11f6977bde5490f5ad99e66100a99387f14db02752a38d23ce5fa573d411 WatchSource:0}: Error finding container b8fe11f6977bde5490f5ad99e66100a99387f14db02752a38d23ce5fa573d411: Status 404 returned error can't find the container with id b8fe11f6977bde5490f5ad99e66100a99387f14db02752a38d23ce5fa573d411
Jul 24 22:57:25.503483 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:25.503404    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:26.133615 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:26.133413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww" event={"ID":"3185626a-2701-4b83-9807-58def62c683f","Type":"ContainerStarted","Data":"b8fe11f6977bde5490f5ad99e66100a99387f14db02752a38d23ce5fa573d411"}
Jul 24 22:57:29.139973 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:29.139933    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 22:57:29.140278 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:29.140022    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 22:57:30.159060 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.159026    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9" event={"ID":"d22bdb04-157a-4ccd-bab8-c672428aada2","Type":"ContainerStarted","Data":"3bfae9c9047868febd839cf95c89469c64ca28f37c6d70d3f9bf0499a383b435"}
Jul 24 22:57:30.160718 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.160694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" event={"ID":"b10a7454-9520-49eb-8325-a8ece7cfc664","Type":"ContainerStarted","Data":"d4ba6558e58eca105c312c415d499b3a081b77dc6bcbc8d2d8794119602cce89"}
Jul 24 22:57:30.160853 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.160820    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:30.162772 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.162728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr" event={"ID":"a36ff974-d2d8-4de7-ac17-b7885415da66","Type":"ContainerStarted","Data":"77eb6928c015f508cafcdb3c8e1d986b0d990b225e4670cbc4d3fe1cea2f3859"}
Jul 24 22:57:30.162898 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.162880    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:30.163014 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.162943    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-phrvz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 22:57:30.163014 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.162981    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 22:57:30.164784 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.164765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww" event={"ID":"3185626a-2701-4b83-9807-58def62c683f","Type":"ContainerStarted","Data":"44dcdabbc89f810b794273f451788b3e313650c90ea8ae0146008fafba07e97c"}
Jul 24 22:57:30.164947 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.164926    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:57:30.185983 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.185927    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-745f8948c6-bt5q9" podStartSLOduration=2.597521585 podStartE2EDuration="7.185912346s" totalImagesPullingTime="4.588390761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:30.184616856 +0000 UTC m=+311.037943420" watchObservedRunningTime="2026-07-24 22:57:30.185912346 +0000 UTC m=+311.039238903"
Jul 24 22:57:30.209082 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.209021    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr" podStartSLOduration=2.631693234 podStartE2EDuration="7.208999078s" totalImagesPullingTime="4.577305844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:30.206571792 +0000 UTC m=+311.059898331" watchObservedRunningTime="2026-07-24 22:57:30.208999078 +0000 UTC m=+311.062325619"
Jul 24 22:57:30.210896 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.210868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:30.233671 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.233617    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww" podStartSLOduration=2.428438871 podStartE2EDuration="6.233602s" totalImagesPullingTime="3.805163129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:30.232421736 +0000 UTC m=+311.085748277" watchObservedRunningTime="2026-07-24 22:57:30.233602 +0000 UTC m=+311.086928539"
Jul 24 22:57:30.256182 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.256114    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" podStartSLOduration=2.190776952 podStartE2EDuration="6.256079723s" totalImagesPullingTime="4.065302771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:30.254219828 +0000 UTC m=+311.107546392" watchObservedRunningTime="2026-07-24 22:57:30.256079723 +0000 UTC m=+311.109406270"
Jul 24 22:57:30.302592 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.302558    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:30.514199 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.514092    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:30.670569 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.670536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:30.826550 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.826482    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"]
Jul 24 22:57:30.830012 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.829993    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.832195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.832176    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-j6cww\""
Jul 24 22:57:30.852850 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.852828    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"]
Jul 24 22:57:30.874504 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.874482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:30.896028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896108 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896108 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896108 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896257 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896257 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwgls\" (UniqueName: \"kubernetes.io/projected/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-kube-api-access-lwgls\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896257 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896355 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.896355 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.896323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997590 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997595    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997747 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997706    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997747 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997738    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997839 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997839 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997839 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwgls\" (UniqueName: \"kubernetes.io/projected/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-kube-api-access-lwgls\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.997955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.997907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.998101 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.998010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.998194 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.998095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.998349 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.998319    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.998461 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.998381    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.998461 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.998444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:30.999958 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:30.999939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:31.000222 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:31.000204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:31.006450 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:31.006432    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:31.007402 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:31.007377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwgls\" (UniqueName: \"kubernetes.io/projected/2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad-kube-api-access-lwgls\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft\" (UID: \"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:31.140104 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:31.140080    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:31.170986 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:31.170960    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:31.322869 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:31.322848    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"]
Jul 24 22:57:31.324100 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:31.324078    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9c4c56_d1b4_40ff_8b5f_e8f6232520ad.slice/crio-56730af597ab1db06ccb40f95a06907246e03baa674d4a242796b24b6134d7c3 WatchSource:0}: Error finding container 56730af597ab1db06ccb40f95a06907246e03baa674d4a242796b24b6134d7c3: Status 404 returned error can't find the container with id 56730af597ab1db06ccb40f95a06907246e03baa674d4a242796b24b6134d7c3
Jul 24 22:57:32.177176 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:32.177111    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft" event={"ID":"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad","Type":"ContainerStarted","Data":"56730af597ab1db06ccb40f95a06907246e03baa674d4a242796b24b6134d7c3"}
Jul 24 22:57:33.729593 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.729507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:33.818253 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.817708    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"]
Jul 24 22:57:33.821516 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.821478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:33.821655 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.821482    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"]
Jul 24 22:57:33.825240 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.825209    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-d75l2\""
Jul 24 22:57:33.825853 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.825330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:33.825853 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.825788    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 22:57:33.828107 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.827938    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4cgtp\""
Jul 24 22:57:33.830753 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.830734    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 22:57:33.838353 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.838331    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"]
Jul 24 22:57:33.847630 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.847607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"]
Jul 24 22:57:33.862000 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.861907    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:33.926950 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.926924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc715d7e-142f-4c70-a2c9-86cbd9077540-cert\") pod \"kserve-controller-manager-d6f4b65b8-jr2bk\" (UID: \"bc715d7e-142f-4c70-a2c9-86cbd9077540\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:33.927069 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.926977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llm67\" (UniqueName: \"kubernetes.io/projected/bc715d7e-142f-4c70-a2c9-86cbd9077540-kube-api-access-llm67\") pod \"kserve-controller-manager-d6f4b65b8-jr2bk\" (UID: \"bc715d7e-142f-4c70-a2c9-86cbd9077540\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:33.927069 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.927000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95265a02-5c3d-4466-90cb-d295ef074376-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r4q4f\" (UID: \"95265a02-5c3d-4466-90cb-d295ef074376\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:33.927069 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.927027    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlzj4\" (UniqueName: \"kubernetes.io/projected/95265a02-5c3d-4466-90cb-d295ef074376-kube-api-access-tlzj4\") pod \"llmisvc-controller-manager-777c6bd7d5-r4q4f\" (UID: \"95265a02-5c3d-4466-90cb-d295ef074376\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:33.987405 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:33.987326    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:34.028415 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.028391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc715d7e-142f-4c70-a2c9-86cbd9077540-cert\") pod \"kserve-controller-manager-d6f4b65b8-jr2bk\" (UID: \"bc715d7e-142f-4c70-a2c9-86cbd9077540\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:34.028575 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.028445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llm67\" (UniqueName: \"kubernetes.io/projected/bc715d7e-142f-4c70-a2c9-86cbd9077540-kube-api-access-llm67\") pod \"kserve-controller-manager-d6f4b65b8-jr2bk\" (UID: \"bc715d7e-142f-4c70-a2c9-86cbd9077540\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:34.028575 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.028468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95265a02-5c3d-4466-90cb-d295ef074376-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r4q4f\" (UID: \"95265a02-5c3d-4466-90cb-d295ef074376\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:34.028575 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.028495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlzj4\" (UniqueName: \"kubernetes.io/projected/95265a02-5c3d-4466-90cb-d295ef074376-kube-api-access-tlzj4\") pod \"llmisvc-controller-manager-777c6bd7d5-r4q4f\" (UID: \"95265a02-5c3d-4466-90cb-d295ef074376\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:34.031225 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.031196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95265a02-5c3d-4466-90cb-d295ef074376-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r4q4f\" (UID: \"95265a02-5c3d-4466-90cb-d295ef074376\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:34.031225 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.031213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc715d7e-142f-4c70-a2c9-86cbd9077540-cert\") pod \"kserve-controller-manager-d6f4b65b8-jr2bk\" (UID: \"bc715d7e-142f-4c70-a2c9-86cbd9077540\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:34.040332 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.040306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llm67\" (UniqueName: \"kubernetes.io/projected/bc715d7e-142f-4c70-a2c9-86cbd9077540-kube-api-access-llm67\") pod \"kserve-controller-manager-d6f4b65b8-jr2bk\" (UID: \"bc715d7e-142f-4c70-a2c9-86cbd9077540\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:34.040623 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.040586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlzj4\" (UniqueName: \"kubernetes.io/projected/95265a02-5c3d-4466-90cb-d295ef074376-kube-api-access-tlzj4\") pod \"llmisvc-controller-manager-777c6bd7d5-r4q4f\" (UID: \"95265a02-5c3d-4466-90cb-d295ef074376\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:34.064496 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.064469    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:34.140037 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.140013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:34.147314 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:34.147295    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:35.016625 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.016581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:35.054398 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.054365    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 22:57:35.054510 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.054430    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 22:57:35.054510 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.054456    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 22:57:35.118691 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.118656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:57:35.120118 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.120096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"]
Jul 24 22:57:35.122624 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:35.122600    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod95265a02_5c3d_4466_90cb_d295ef074376.slice/crio-1f87246cb10f5493756338f427eb19887c6234d58053958a67b3aba0e46ad971 WatchSource:0}: Error finding container 1f87246cb10f5493756338f427eb19887c6234d58053958a67b3aba0e46ad971: Status 404 returned error can't find the container with id 1f87246cb10f5493756338f427eb19887c6234d58053958a67b3aba0e46ad971
Jul 24 22:57:35.162087 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.162063    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"]
Jul 24 22:57:35.163820 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:57:35.163775    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc715d7e_142f_4c70_a2c9_86cbd9077540.slice/crio-1228233bff41db6c699f5d3484b83fc1b834b3066fe555a50024766be445e9dc WatchSource:0}: Error finding container 1228233bff41db6c699f5d3484b83fc1b834b3066fe555a50024766be445e9dc: Status 404 returned error can't find the container with id 1228233bff41db6c699f5d3484b83fc1b834b3066fe555a50024766be445e9dc
Jul 24 22:57:35.199373 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.199346    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk" event={"ID":"bc715d7e-142f-4c70-a2c9-86cbd9077540","Type":"ContainerStarted","Data":"1228233bff41db6c699f5d3484b83fc1b834b3066fe555a50024766be445e9dc"}
Jul 24 22:57:35.200843 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.200821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft" event={"ID":"2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad","Type":"ContainerStarted","Data":"41e4fabe87c113b61c1695075136a5c9d8ab7bce449def5b1c1951cfe73ed3d5"}
Jul 24 22:57:35.202083 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.202058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f" event={"ID":"95265a02-5c3d-4466-90cb-d295ef074376","Type":"ContainerStarted","Data":"1f87246cb10f5493756338f427eb19887c6234d58053958a67b3aba0e46ad971"}
Jul 24 22:57:35.244689 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:35.244597    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft" podStartSLOduration=1.5165747710000002 podStartE2EDuration="5.244579136s" totalImagesPullingTime="3.728004365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:35.24330099 +0000 UTC m=+316.096627532" watchObservedRunningTime="2026-07-24 22:57:35.244579136 +0000 UTC m=+316.097905676"
Jul 24 22:57:36.141019 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:36.140988    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:36.145766 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:36.145745    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:36.208748 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:36.208682    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:36.209965 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:36.209941    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft"
Jul 24 22:57:36.707133 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:36.707099    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"]
Jul 24 22:57:36.707351 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:36.707328    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerName="discovery" containerID="cri-o://d4ba6558e58eca105c312c415d499b3a081b77dc6bcbc8d2d8794119602cce89" gracePeriod=30
Jul 24 22:57:37.170679 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:37.170640    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-phrvz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:8080/ready\": dial tcp 10.132.0.35:8080: connect: connection refused" start-of-body=
Jul 24 22:57:37.171143 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:37.170707    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.35:8080/ready\": dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 24 22:57:37.216846 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:37.216813    2564 generic.go:356] "Generic (PLEG): container finished" podID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerID="d4ba6558e58eca105c312c415d499b3a081b77dc6bcbc8d2d8794119602cce89" exitCode=0
Jul 24 22:57:37.217040 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:37.216884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" event={"ID":"b10a7454-9520-49eb-8325-a8ece7cfc664","Type":"ContainerDied","Data":"d4ba6558e58eca105c312c415d499b3a081b77dc6bcbc8d2d8794119602cce89"}
Jul 24 22:57:37.928701 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:37.928680    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:38.066975 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.066946    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htqgp\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-kube-api-access-htqgp\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067328 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-ca-configmap\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067328 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067045    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10a7454-9520-49eb-8325-a8ece7cfc664-local-certs\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067328 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067078    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-cacerts\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067328 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067114    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-token\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067328 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067207    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-dns-cert\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067328 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067256    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-kubeconfig\") pod \"b10a7454-9520-49eb-8325-a8ece7cfc664\" (UID: \"b10a7454-9520-49eb-8325-a8ece7cfc664\") "
Jul 24 22:57:38.067725 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067386    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:57:38.067725 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.067604    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-ca-configmap\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.069716 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.069685    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:57:38.069817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.069709    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-cacerts" (OuterVolumeSpecName: "cacerts") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:57:38.069817 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.069714    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-token" (OuterVolumeSpecName: "istio-token") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:57:38.069932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.069830    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-kube-api-access-htqgp" (OuterVolumeSpecName: "kube-api-access-htqgp") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "kube-api-access-htqgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:57:38.069932 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.069882    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:57:38.070028 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.070009    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b10a7454-9520-49eb-8325-a8ece7cfc664-local-certs" (OuterVolumeSpecName: "local-certs") pod "b10a7454-9520-49eb-8325-a8ece7cfc664" (UID: "b10a7454-9520-49eb-8325-a8ece7cfc664"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:57:38.168845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.168773    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-kubeconfig\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.168845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.168800    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htqgp\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-kube-api-access-htqgp\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.168845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.168810    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10a7454-9520-49eb-8325-a8ece7cfc664-local-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.168845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.168819    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-cacerts\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.168845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.168827    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-token\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.168845 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.168836    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b10a7454-9520-49eb-8325-a8ece7cfc664-istio-csr-dns-cert\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:57:38.228540 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.228504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f" event={"ID":"95265a02-5c3d-4466-90cb-d295ef074376","Type":"ContainerStarted","Data":"3170004a33b700a7f14f46620a93fe0be82b48957452e849482ef0be59ee45f5"}
Jul 24 22:57:38.229019 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.228616    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:57:38.230332 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.230309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk" event={"ID":"bc715d7e-142f-4c70-a2c9-86cbd9077540","Type":"ContainerStarted","Data":"bffebcbd4298c349d1caf8a4d0c2b95b023bf49c4aaa8648ff734d68703de7ac"}
Jul 24 22:57:38.230440 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.230350    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:57:38.231734 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.231700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz" event={"ID":"b10a7454-9520-49eb-8325-a8ece7cfc664","Type":"ContainerDied","Data":"328e17138eee4759552a263149a793a48a417cbd10ff945fc7a2d4737862e806"}
Jul 24 22:57:38.231811 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.231761    2564 scope.go:117] "RemoveContainer" containerID="d4ba6558e58eca105c312c415d499b3a081b77dc6bcbc8d2d8794119602cce89"
Jul 24 22:57:38.231811 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.231719    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"
Jul 24 22:57:38.263777 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.263734    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f" podStartSLOduration=2.44204282 podStartE2EDuration="5.263721107s" totalImagesPullingTime="2.821678287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:38.258751567 +0000 UTC m=+319.112078107" watchObservedRunningTime="2026-07-24 22:57:38.263721107 +0000 UTC m=+319.117047645"
Jul 24 22:57:38.279775 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.279749    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"]
Jul 24 22:57:38.283959 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.283939    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-phrvz"]
Jul 24 22:57:38.309181 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:38.309139    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk" podStartSLOduration=2.533647182 podStartE2EDuration="5.309115847s" totalImagesPullingTime="2.775468665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:38.307407374 +0000 UTC m=+319.160733914" watchObservedRunningTime="2026-07-24 22:57:38.309115847 +0000 UTC m=+319.162442386"
Jul 24 22:57:39.694636 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:39.694606    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" path="/var/lib/kubelet/pods/b10a7454-9520-49eb-8325-a8ece7cfc664/volumes"
Jul 24 22:57:41.173106 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:41.173081    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-vthzr"
Jul 24 22:57:41.175979 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:57:41.175956    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-z79ww"
Jul 24 22:58:09.240318 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:09.240285    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r4q4f"
Jul 24 22:58:09.243571 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:09.243551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jr2bk"
Jul 24 22:58:24.163038 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.162997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:24.260000 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.259956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:24.296022 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.295987    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk9px"]
Jul 24 22:58:24.296366 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.296354    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerName="discovery"
Jul 24 22:58:24.296412 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.296368    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerName="discovery"
Jul 24 22:58:24.296493 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.296424    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10a7454-9520-49eb-8325-a8ece7cfc664" containerName="discovery"
Jul 24 22:58:24.300712 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.300688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:24.302760 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.302735    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5dg96\""
Jul 24 22:58:24.302888 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.302815    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 22:58:24.303509 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.303493    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 22:58:24.307037 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.306984    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk9px"]
Jul 24 22:58:24.325474 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.325448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg6nm\" (UniqueName: \"kubernetes.io/projected/82f0a5a6-125d-4f94-85b1-9dc24c80ae33-kube-api-access-bg6nm\") pod \"kuadrant-operator-catalog-lk9px\" (UID: \"82f0a5a6-125d-4f94-85b1-9dc24c80ae33\") " pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:24.338990 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.338964    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:24.426043 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.425959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg6nm\" (UniqueName: \"kubernetes.io/projected/82f0a5a6-125d-4f94-85b1-9dc24c80ae33-kube-api-access-bg6nm\") pod \"kuadrant-operator-catalog-lk9px\" (UID: \"82f0a5a6-125d-4f94-85b1-9dc24c80ae33\") " pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:24.433520 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.433487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg6nm\" (UniqueName: \"kubernetes.io/projected/82f0a5a6-125d-4f94-85b1-9dc24c80ae33-kube-api-access-bg6nm\") pod \"kuadrant-operator-catalog-lk9px\" (UID: \"82f0a5a6-125d-4f94-85b1-9dc24c80ae33\") " pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:24.612810 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.612778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:24.655945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.655914    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk9px"]
Jul 24 22:58:24.744047 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.744016    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk9px"]
Jul 24 22:58:24.746220 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:58:24.746189    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f0a5a6_125d_4f94_85b1_9dc24c80ae33.slice/crio-344c102a6df67169dfe3b407b7db4ae0fd95a038a938ec67b2a9dfa93c98817c WatchSource:0}: Error finding container 344c102a6df67169dfe3b407b7db4ae0fd95a038a938ec67b2a9dfa93c98817c: Status 404 returned error can't find the container with id 344c102a6df67169dfe3b407b7db4ae0fd95a038a938ec67b2a9dfa93c98817c
Jul 24 22:58:24.865979 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.865945    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lvbvc"]
Jul 24 22:58:24.870295 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.870281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:24.875833 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.875809    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lvbvc"]
Jul 24 22:58:24.930117 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:24.930090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssczk\" (UniqueName: \"kubernetes.io/projected/472a249b-9d6a-4999-b9b8-91a0d0ea31d9-kube-api-access-ssczk\") pod \"kuadrant-operator-catalog-lvbvc\" (UID: \"472a249b-9d6a-4999-b9b8-91a0d0ea31d9\") " pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:25.033501 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:25.033415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssczk\" (UniqueName: \"kubernetes.io/projected/472a249b-9d6a-4999-b9b8-91a0d0ea31d9-kube-api-access-ssczk\") pod \"kuadrant-operator-catalog-lvbvc\" (UID: \"472a249b-9d6a-4999-b9b8-91a0d0ea31d9\") " pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:25.040608 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:25.040586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssczk\" (UniqueName: \"kubernetes.io/projected/472a249b-9d6a-4999-b9b8-91a0d0ea31d9-kube-api-access-ssczk\") pod \"kuadrant-operator-catalog-lvbvc\" (UID: \"472a249b-9d6a-4999-b9b8-91a0d0ea31d9\") " pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:25.180269 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:25.180244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:25.313491 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:25.313457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lvbvc"]
Jul 24 22:58:25.383798 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:58:25.383756    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472a249b_9d6a_4999_b9b8_91a0d0ea31d9.slice/crio-15e12f0db38d9582f41431067e1f5f6359750e2fee8727a942dddd83e92f0814 WatchSource:0}: Error finding container 15e12f0db38d9582f41431067e1f5f6359750e2fee8727a942dddd83e92f0814: Status 404 returned error can't find the container with id 15e12f0db38d9582f41431067e1f5f6359750e2fee8727a942dddd83e92f0814
Jul 24 22:58:25.465619 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:25.465588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk9px" event={"ID":"82f0a5a6-125d-4f94-85b1-9dc24c80ae33","Type":"ContainerStarted","Data":"344c102a6df67169dfe3b407b7db4ae0fd95a038a938ec67b2a9dfa93c98817c"}
Jul 24 22:58:25.466861 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:25.466838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc" event={"ID":"472a249b-9d6a-4999-b9b8-91a0d0ea31d9","Type":"ContainerStarted","Data":"15e12f0db38d9582f41431067e1f5f6359750e2fee8727a942dddd83e92f0814"}
Jul 24 22:58:27.479187 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.479154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk9px" event={"ID":"82f0a5a6-125d-4f94-85b1-9dc24c80ae33","Type":"ContainerStarted","Data":"8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7"}
Jul 24 22:58:27.479632 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.479213    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-lk9px" podUID="82f0a5a6-125d-4f94-85b1-9dc24c80ae33" containerName="registry-server" containerID="cri-o://8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7" gracePeriod=2
Jul 24 22:58:27.480724 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.480702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc" event={"ID":"472a249b-9d6a-4999-b9b8-91a0d0ea31d9","Type":"ContainerStarted","Data":"4ac3e3b76dd4a85d6cf9c9c1507aa3fb69a590d7d8100fc5f52ad116b6a24bd4"}
Jul 24 22:58:27.494550 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.494505    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lk9px" podStartSLOduration=1.663200578 podStartE2EDuration="3.494492351s" totalImagesPullingTime="1.831291773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:58:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:58:27.493444683 +0000 UTC m=+368.346771224" watchObservedRunningTime="2026-07-24 22:58:27.494492351 +0000 UTC m=+368.347818890"
Jul 24 22:58:27.508279 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.508240    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc" podStartSLOduration=2.313631594 podStartE2EDuration="3.508223949s" totalImagesPullingTime="1.194592355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:58:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:58:27.507623062 +0000 UTC m=+368.360949602" watchObservedRunningTime="2026-07-24 22:58:27.508223949 +0000 UTC m=+368.361550499"
Jul 24 22:58:27.726834 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.726813    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:27.758856 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.758790    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg6nm\" (UniqueName: \"kubernetes.io/projected/82f0a5a6-125d-4f94-85b1-9dc24c80ae33-kube-api-access-bg6nm\") pod \"82f0a5a6-125d-4f94-85b1-9dc24c80ae33\" (UID: \"82f0a5a6-125d-4f94-85b1-9dc24c80ae33\") "
Jul 24 22:58:27.760823 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.760794    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82f0a5a6-125d-4f94-85b1-9dc24c80ae33-kube-api-access-bg6nm" (OuterVolumeSpecName: "kube-api-access-bg6nm") pod "82f0a5a6-125d-4f94-85b1-9dc24c80ae33" (UID: "82f0a5a6-125d-4f94-85b1-9dc24c80ae33"). InnerVolumeSpecName "kube-api-access-bg6nm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:58:27.860260 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:27.860223    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg6nm\" (UniqueName: \"kubernetes.io/projected/82f0a5a6-125d-4f94-85b1-9dc24c80ae33-kube-api-access-bg6nm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:28.486488 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.486457    2564 generic.go:356] "Generic (PLEG): container finished" podID="82f0a5a6-125d-4f94-85b1-9dc24c80ae33" containerID="8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7" exitCode=0
Jul 24 22:58:28.486955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.486509    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk9px"
Jul 24 22:58:28.486955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.486534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk9px" event={"ID":"82f0a5a6-125d-4f94-85b1-9dc24c80ae33","Type":"ContainerDied","Data":"8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7"}
Jul 24 22:58:28.486955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.486579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk9px" event={"ID":"82f0a5a6-125d-4f94-85b1-9dc24c80ae33","Type":"ContainerDied","Data":"344c102a6df67169dfe3b407b7db4ae0fd95a038a938ec67b2a9dfa93c98817c"}
Jul 24 22:58:28.486955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.486601    2564 scope.go:117] "RemoveContainer" containerID="8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7"
Jul 24 22:58:28.495441 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.495424    2564 scope.go:117] "RemoveContainer" containerID="8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7"
Jul 24 22:58:28.495692 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:58:28.495676    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7\": container with ID starting with 8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7 not found: ID does not exist" containerID="8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7"
Jul 24 22:58:28.495737 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.495700    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7"} err="failed to get container status \"8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7\": rpc error: code = NotFound desc = could not find container \"8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7\": container with ID starting with 8f6c8a293c921d50f6109f8c026ef2b532ebfc051e2e25af12578dd3f2a04ab7 not found: ID does not exist"
Jul 24 22:58:28.506560 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.506537    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk9px"]
Jul 24 22:58:28.509515 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.509495    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk9px"]
Jul 24 22:58:28.947141 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:28.947102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:29.048732 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:29.048698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:29.694835 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:29.694802    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82f0a5a6-125d-4f94-85b1-9dc24c80ae33" path="/var/lib/kubelet/pods/82f0a5a6-125d-4f94-85b1-9dc24c80ae33/volumes"
Jul 24 22:58:30.692688 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:30.692659    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:35.180420 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:35.180388    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:35.180834 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:35.180529    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:35.201314 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:35.201290    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:35.540712 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:35.540648    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lvbvc"
Jul 24 22:58:47.927941 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.927902    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"]
Jul 24 22:58:47.928470 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.928452    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f0a5a6-125d-4f94-85b1-9dc24c80ae33" containerName="registry-server"
Jul 24 22:58:47.928551 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.928473    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f0a5a6-125d-4f94-85b1-9dc24c80ae33" containerName="registry-server"
Jul 24 22:58:47.928610 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.928602    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f0a5a6-125d-4f94-85b1-9dc24c80ae33" containerName="registry-server"
Jul 24 22:58:47.931942 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.931921    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:47.934231 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.934214    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6p7bk\""
Jul 24 22:58:47.939155 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:47.939111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"]
Jul 24 22:58:48.019758 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.019706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.019914 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.019802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.019914 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.019831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxnxx\" (UniqueName: \"kubernetes.io/projected/02959104-3c14-4720-b78e-040bea03526d-kube-api-access-hxnxx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.030935 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.030912    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"]
Jul 24 22:58:48.034502 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.034486    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.041947 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.041926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"]
Jul 24 22:58:48.120818 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.120795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.120818 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.120820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxnxx\" (UniqueName: \"kubernetes.io/projected/02959104-3c14-4720-b78e-040bea03526d-kube-api-access-hxnxx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.120968 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.120841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.120968 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.120872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.121061 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.121000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.121061 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.121033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mx9v\" (UniqueName: \"kubernetes.io/projected/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-kube-api-access-8mx9v\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.121222 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.121206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.121296 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.121281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.128629 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.128608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxnxx\" (UniqueName: \"kubernetes.io/projected/02959104-3c14-4720-b78e-040bea03526d-kube-api-access-hxnxx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.221827 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.221757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mx9v\" (UniqueName: \"kubernetes.io/projected/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-kube-api-access-8mx9v\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.221827 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.221807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.221982 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.221855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.222210 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.222192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.222269 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.222233    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.229924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.229900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mx9v\" (UniqueName: \"kubernetes.io/projected/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-kube-api-access-8mx9v\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.242299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.242278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:48.329160 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.329117    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"]
Jul 24 22:58:48.334547 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.334524    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.339941 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.339917    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"]
Jul 24 22:58:48.343666 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.343641    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:48.375868 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.375846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"]
Jul 24 22:58:48.377688 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:58:48.377666    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02959104_3c14_4720_b78e_040bea03526d.slice/crio-a9fb025c74a7fe3dc95c6456d97f078f99fe5152ef1291fbc82120d92a53186a WatchSource:0}: Error finding container a9fb025c74a7fe3dc95c6456d97f078f99fe5152ef1291fbc82120d92a53186a: Status 404 returned error can't find the container with id a9fb025c74a7fe3dc95c6456d97f078f99fe5152ef1291fbc82120d92a53186a
Jul 24 22:58:48.422961 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.422877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp5cd\" (UniqueName: \"kubernetes.io/projected/90698fdb-a1f9-45ac-99f5-3a08112ac645-kube-api-access-hp5cd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.423064 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.422969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.423064 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.423011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.480860 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.480838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"]
Jul 24 22:58:48.482459 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:58:48.482428    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76b97e5_dad6_4ea0_bba7_e7b2db4d78d6.slice/crio-3dc72c2b79bac3f12e3f4ef6d4a336aaf1fdd9884598544c70f7fb89987c646b WatchSource:0}: Error finding container 3dc72c2b79bac3f12e3f4ef6d4a336aaf1fdd9884598544c70f7fb89987c646b: Status 404 returned error can't find the container with id 3dc72c2b79bac3f12e3f4ef6d4a336aaf1fdd9884598544c70f7fb89987c646b
Jul 24 22:58:48.523451 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.523426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp5cd\" (UniqueName: \"kubernetes.io/projected/90698fdb-a1f9-45ac-99f5-3a08112ac645-kube-api-access-hp5cd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.523540 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.523490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.523540 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.523525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.523862 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.523841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.523945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.523910    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.531354 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.531329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp5cd\" (UniqueName: \"kubernetes.io/projected/90698fdb-a1f9-45ac-99f5-3a08112ac645-kube-api-access-hp5cd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.584296 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.584270    2564 generic.go:356] "Generic (PLEG): container finished" podID="02959104-3c14-4720-b78e-040bea03526d" containerID="e4dbea8b9152d145594334921b7227e9897a9577c7c0d00395187bd9a1931b62" exitCode=0
Jul 24 22:58:48.584431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.584366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz" event={"ID":"02959104-3c14-4720-b78e-040bea03526d","Type":"ContainerDied","Data":"e4dbea8b9152d145594334921b7227e9897a9577c7c0d00395187bd9a1931b62"}
Jul 24 22:58:48.584431 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.584421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz" event={"ID":"02959104-3c14-4720-b78e-040bea03526d","Type":"ContainerStarted","Data":"a9fb025c74a7fe3dc95c6456d97f078f99fe5152ef1291fbc82120d92a53186a"}
Jul 24 22:58:48.586321 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.586291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9" event={"ID":"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6","Type":"ContainerStarted","Data":"56de1ddea5862902bf3273ca82f88cbc1a65680b809c701d489dbcbe37e7e5d1"}
Jul 24 22:58:48.586406 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.586328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9" event={"ID":"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6","Type":"ContainerStarted","Data":"3dc72c2b79bac3f12e3f4ef6d4a336aaf1fdd9884598544c70f7fb89987c646b"}
Jul 24 22:58:48.646397 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.646366    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:48.731390 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.731320    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"]
Jul 24 22:58:48.737920 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.737894    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.740715 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.740695    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"]
Jul 24 22:58:48.774955 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.774935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"]
Jul 24 22:58:48.776406 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:58:48.776382    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90698fdb_a1f9_45ac_99f5_3a08112ac645.slice/crio-19321907ea9f98f1fd5556fe0e2830c0dd57b54ef381acaa170fd3ee550a98cc WatchSource:0}: Error finding container 19321907ea9f98f1fd5556fe0e2830c0dd57b54ef381acaa170fd3ee550a98cc: Status 404 returned error can't find the container with id 19321907ea9f98f1fd5556fe0e2830c0dd57b54ef381acaa170fd3ee550a98cc
Jul 24 22:58:48.827046 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.827025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.827174 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.827053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.827174 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.827077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9r2p\" (UniqueName: \"kubernetes.io/projected/7bb24c70-ab49-4285-b359-c278085e63e4-kube-api-access-z9r2p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.928413 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.928389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.928777 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.928421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.928777 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.928449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9r2p\" (UniqueName: \"kubernetes.io/projected/7bb24c70-ab49-4285-b359-c278085e63e4-kube-api-access-z9r2p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.928777 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.928734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.928777 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.928763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:48.936258 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:48.936239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9r2p\" (UniqueName: \"kubernetes.io/projected/7bb24c70-ab49-4285-b359-c278085e63e4-kube-api-access-z9r2p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:49.050459 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.050378    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:49.181755 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.181728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"]
Jul 24 22:58:49.191054 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:58:49.191027    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb24c70_ab49_4285_b359_c278085e63e4.slice/crio-b8ba6a9228c6ec00ab80039867c5920cb444b6ebca05fe91b3475919b56f06ac WatchSource:0}: Error finding container b8ba6a9228c6ec00ab80039867c5920cb444b6ebca05fe91b3475919b56f06ac: Status 404 returned error can't find the container with id b8ba6a9228c6ec00ab80039867c5920cb444b6ebca05fe91b3475919b56f06ac
Jul 24 22:58:49.593035 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.593000    2564 generic.go:356] "Generic (PLEG): container finished" podID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerID="56de1ddea5862902bf3273ca82f88cbc1a65680b809c701d489dbcbe37e7e5d1" exitCode=0
Jul 24 22:58:49.593225 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.593084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9" event={"ID":"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6","Type":"ContainerDied","Data":"56de1ddea5862902bf3273ca82f88cbc1a65680b809c701d489dbcbe37e7e5d1"}
Jul 24 22:58:49.594656 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.594632    2564 generic.go:356] "Generic (PLEG): container finished" podID="7bb24c70-ab49-4285-b359-c278085e63e4" containerID="10b80d99de4333872d1c5f5dcabaa0a0ea5cf34c0e568ddb1f6f866627a69212" exitCode=0
Jul 24 22:58:49.594816 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.594716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb" event={"ID":"7bb24c70-ab49-4285-b359-c278085e63e4","Type":"ContainerDied","Data":"10b80d99de4333872d1c5f5dcabaa0a0ea5cf34c0e568ddb1f6f866627a69212"}
Jul 24 22:58:49.594816 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.594763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb" event={"ID":"7bb24c70-ab49-4285-b359-c278085e63e4","Type":"ContainerStarted","Data":"b8ba6a9228c6ec00ab80039867c5920cb444b6ebca05fe91b3475919b56f06ac"}
Jul 24 22:58:49.596279 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.596260    2564 generic.go:356] "Generic (PLEG): container finished" podID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerID="c47c5125ec909d9fb58e8c2027cded419b600a60ede32ad1683a3d0509508a30" exitCode=0
Jul 24 22:58:49.596379 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.596340    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7" event={"ID":"90698fdb-a1f9-45ac-99f5-3a08112ac645","Type":"ContainerDied","Data":"c47c5125ec909d9fb58e8c2027cded419b600a60ede32ad1683a3d0509508a30"}
Jul 24 22:58:49.596445 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.596423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7" event={"ID":"90698fdb-a1f9-45ac-99f5-3a08112ac645","Type":"ContainerStarted","Data":"19321907ea9f98f1fd5556fe0e2830c0dd57b54ef381acaa170fd3ee550a98cc"}
Jul 24 22:58:49.598397 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.598377    2564 generic.go:356] "Generic (PLEG): container finished" podID="02959104-3c14-4720-b78e-040bea03526d" containerID="ff976e6bb73adc1e263e023ad4614d7d4dfaf9177304c767d0ac7204fe9343ed" exitCode=0
Jul 24 22:58:49.598512 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:49.598445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz" event={"ID":"02959104-3c14-4720-b78e-040bea03526d","Type":"ContainerDied","Data":"ff976e6bb73adc1e263e023ad4614d7d4dfaf9177304c767d0ac7204fe9343ed"}
Jul 24 22:58:50.605675 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.605647    2564 generic.go:356] "Generic (PLEG): container finished" podID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerID="586114d7a87ef9d25e7ee1a6c55dc24ad1ec4d1b57eab8b670f43ef484968109" exitCode=0
Jul 24 22:58:50.606166 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.605719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9" event={"ID":"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6","Type":"ContainerDied","Data":"586114d7a87ef9d25e7ee1a6c55dc24ad1ec4d1b57eab8b670f43ef484968109"}
Jul 24 22:58:50.607796 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.607770    2564 generic.go:356] "Generic (PLEG): container finished" podID="7bb24c70-ab49-4285-b359-c278085e63e4" containerID="ec14045c707e4270b6aa4e5b2c989c0c42ea94f8a803ac68f1c31867b1495e5b" exitCode=0
Jul 24 22:58:50.607900 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.607872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb" event={"ID":"7bb24c70-ab49-4285-b359-c278085e63e4","Type":"ContainerDied","Data":"ec14045c707e4270b6aa4e5b2c989c0c42ea94f8a803ac68f1c31867b1495e5b"}
Jul 24 22:58:50.610133 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.610093    2564 generic.go:356] "Generic (PLEG): container finished" podID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerID="7ec6090d8a7cff210f135180f068b90671662eeb8f79dc345f81e87f41dddfd1" exitCode=0
Jul 24 22:58:50.610242 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.610163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7" event={"ID":"90698fdb-a1f9-45ac-99f5-3a08112ac645","Type":"ContainerDied","Data":"7ec6090d8a7cff210f135180f068b90671662eeb8f79dc345f81e87f41dddfd1"}
Jul 24 22:58:50.613152 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.613080    2564 generic.go:356] "Generic (PLEG): container finished" podID="02959104-3c14-4720-b78e-040bea03526d" containerID="84976d2eafbce76fc2eba7d401cc4661759b162f7e8d29f23346d61ad271b61b" exitCode=0
Jul 24 22:58:50.613241 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:50.613154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz" event={"ID":"02959104-3c14-4720-b78e-040bea03526d","Type":"ContainerDied","Data":"84976d2eafbce76fc2eba7d401cc4661759b162f7e8d29f23346d61ad271b61b"}
Jul 24 22:58:51.620740 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.620712    2564 generic.go:356] "Generic (PLEG): container finished" podID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerID="252883ae9583585a5b8efa57f65365473c5c1336e3b07d3468bdcf1f9c9e72ec" exitCode=0
Jul 24 22:58:51.621119 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.620796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9" event={"ID":"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6","Type":"ContainerDied","Data":"252883ae9583585a5b8efa57f65365473c5c1336e3b07d3468bdcf1f9c9e72ec"}
Jul 24 22:58:51.622719 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.622688    2564 generic.go:356] "Generic (PLEG): container finished" podID="7bb24c70-ab49-4285-b359-c278085e63e4" containerID="ada3ff98ec96bf3488cc87baeb3f92e5d5873ce523e3be5acdf80b667eb270e0" exitCode=0
Jul 24 22:58:51.622847 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.622759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb" event={"ID":"7bb24c70-ab49-4285-b359-c278085e63e4","Type":"ContainerDied","Data":"ada3ff98ec96bf3488cc87baeb3f92e5d5873ce523e3be5acdf80b667eb270e0"}
Jul 24 22:58:51.627860 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.627839    2564 generic.go:356] "Generic (PLEG): container finished" podID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerID="4b3d9e36866bc0ccac28c3375c2dc821f49b10972de75dac4a1c8fb0a1677db2" exitCode=0
Jul 24 22:58:51.627990 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.627918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7" event={"ID":"90698fdb-a1f9-45ac-99f5-3a08112ac645","Type":"ContainerDied","Data":"4b3d9e36866bc0ccac28c3375c2dc821f49b10972de75dac4a1c8fb0a1677db2"}
Jul 24 22:58:51.760444 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.760424    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:51.853805 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.853776    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-bundle\") pod \"02959104-3c14-4720-b78e-040bea03526d\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") "
Jul 24 22:58:51.853934 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.853823    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-util\") pod \"02959104-3c14-4720-b78e-040bea03526d\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") "
Jul 24 22:58:51.853934 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.853846    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxnxx\" (UniqueName: \"kubernetes.io/projected/02959104-3c14-4720-b78e-040bea03526d-kube-api-access-hxnxx\") pod \"02959104-3c14-4720-b78e-040bea03526d\" (UID: \"02959104-3c14-4720-b78e-040bea03526d\") "
Jul 24 22:58:51.854357 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.854329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-bundle" (OuterVolumeSpecName: "bundle") pod "02959104-3c14-4720-b78e-040bea03526d" (UID: "02959104-3c14-4720-b78e-040bea03526d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:51.855970 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.855950    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02959104-3c14-4720-b78e-040bea03526d-kube-api-access-hxnxx" (OuterVolumeSpecName: "kube-api-access-hxnxx") pod "02959104-3c14-4720-b78e-040bea03526d" (UID: "02959104-3c14-4720-b78e-040bea03526d"). InnerVolumeSpecName "kube-api-access-hxnxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:58:51.859340 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.859316    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-util" (OuterVolumeSpecName: "util") pod "02959104-3c14-4720-b78e-040bea03526d" (UID: "02959104-3c14-4720-b78e-040bea03526d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:51.954457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.954402    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:51.954457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.954422    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02959104-3c14-4720-b78e-040bea03526d-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:51.954457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:51.954431    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxnxx\" (UniqueName: \"kubernetes.io/projected/02959104-3c14-4720-b78e-040bea03526d-kube-api-access-hxnxx\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:52.634543 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.634516    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz"
Jul 24 22:58:52.634924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.634547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz" event={"ID":"02959104-3c14-4720-b78e-040bea03526d","Type":"ContainerDied","Data":"a9fb025c74a7fe3dc95c6456d97f078f99fe5152ef1291fbc82120d92a53186a"}
Jul 24 22:58:52.634924 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.634593    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9fb025c74a7fe3dc95c6456d97f078f99fe5152ef1291fbc82120d92a53186a"
Jul 24 22:58:52.799570 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.799550    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:52.802774 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.802757    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:52.831392 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.831376    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:52.964221 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964146    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp5cd\" (UniqueName: \"kubernetes.io/projected/90698fdb-a1f9-45ac-99f5-3a08112ac645-kube-api-access-hp5cd\") pod \"90698fdb-a1f9-45ac-99f5-3a08112ac645\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") "
Jul 24 22:58:52.964221 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964176    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-util\") pod \"7bb24c70-ab49-4285-b359-c278085e63e4\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964227    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-bundle\") pod \"90698fdb-a1f9-45ac-99f5-3a08112ac645\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964264    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-util\") pod \"90698fdb-a1f9-45ac-99f5-3a08112ac645\" (UID: \"90698fdb-a1f9-45ac-99f5-3a08112ac645\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964283    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9r2p\" (UniqueName: \"kubernetes.io/projected/7bb24c70-ab49-4285-b359-c278085e63e4-kube-api-access-z9r2p\") pod \"7bb24c70-ab49-4285-b359-c278085e63e4\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964310    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-bundle\") pod \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964338    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-bundle\") pod \"7bb24c70-ab49-4285-b359-c278085e63e4\" (UID: \"7bb24c70-ab49-4285-b359-c278085e63e4\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964363    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-util\") pod \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") "
Jul 24 22:58:52.964425 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.964410    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mx9v\" (UniqueName: \"kubernetes.io/projected/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-kube-api-access-8mx9v\") pod \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\" (UID: \"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6\") "
Jul 24 22:58:52.965058 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.965030    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-bundle" (OuterVolumeSpecName: "bundle") pod "f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" (UID: "f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:52.965342 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.965034    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-bundle" (OuterVolumeSpecName: "bundle") pod "90698fdb-a1f9-45ac-99f5-3a08112ac645" (UID: "90698fdb-a1f9-45ac-99f5-3a08112ac645"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:52.965606 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.965577    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-bundle" (OuterVolumeSpecName: "bundle") pod "7bb24c70-ab49-4285-b359-c278085e63e4" (UID: "7bb24c70-ab49-4285-b359-c278085e63e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:52.966793 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.966769    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90698fdb-a1f9-45ac-99f5-3a08112ac645-kube-api-access-hp5cd" (OuterVolumeSpecName: "kube-api-access-hp5cd") pod "90698fdb-a1f9-45ac-99f5-3a08112ac645" (UID: "90698fdb-a1f9-45ac-99f5-3a08112ac645"). InnerVolumeSpecName "kube-api-access-hp5cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:58:52.966946 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.966923    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb24c70-ab49-4285-b359-c278085e63e4-kube-api-access-z9r2p" (OuterVolumeSpecName: "kube-api-access-z9r2p") pod "7bb24c70-ab49-4285-b359-c278085e63e4" (UID: "7bb24c70-ab49-4285-b359-c278085e63e4"). InnerVolumeSpecName "kube-api-access-z9r2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:58:52.966995 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.966941    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-kube-api-access-8mx9v" (OuterVolumeSpecName: "kube-api-access-8mx9v") pod "f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" (UID: "f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6"). InnerVolumeSpecName "kube-api-access-8mx9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:58:52.970674 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.970642    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-util" (OuterVolumeSpecName: "util") pod "7bb24c70-ab49-4285-b359-c278085e63e4" (UID: "7bb24c70-ab49-4285-b359-c278085e63e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:52.971237 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.971219    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-util" (OuterVolumeSpecName: "util") pod "f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" (UID: "f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:52.971492 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:52.971475    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-util" (OuterVolumeSpecName: "util") pod "90698fdb-a1f9-45ac-99f5-3a08112ac645" (UID: "90698fdb-a1f9-45ac-99f5-3a08112ac645"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:58:53.065050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065024    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065050 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065046    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9r2p\" (UniqueName: \"kubernetes.io/projected/7bb24c70-ab49-4285-b359-c278085e63e4-kube-api-access-z9r2p\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065056    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065065    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065073    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065081    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mx9v\" (UniqueName: \"kubernetes.io/projected/f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6-kube-api-access-8mx9v\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065089    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp5cd\" (UniqueName: \"kubernetes.io/projected/90698fdb-a1f9-45ac-99f5-3a08112ac645-kube-api-access-hp5cd\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065097    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb24c70-ab49-4285-b359-c278085e63e4-util\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.065230 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.065105    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90698fdb-a1f9-45ac-99f5-3a08112ac645-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:58:53.641811 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.641777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9" event={"ID":"f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6","Type":"ContainerDied","Data":"3dc72c2b79bac3f12e3f4ef6d4a336aaf1fdd9884598544c70f7fb89987c646b"}
Jul 24 22:58:53.642228 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.641831    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3dc72c2b79bac3f12e3f4ef6d4a336aaf1fdd9884598544c70f7fb89987c646b"
Jul 24 22:58:53.642228 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.641787    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9"
Jul 24 22:58:53.643529 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.643510    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb"
Jul 24 22:58:53.643529 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.643520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb" event={"ID":"7bb24c70-ab49-4285-b359-c278085e63e4","Type":"ContainerDied","Data":"b8ba6a9228c6ec00ab80039867c5920cb444b6ebca05fe91b3475919b56f06ac"}
Jul 24 22:58:53.643693 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.643560    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8ba6a9228c6ec00ab80039867c5920cb444b6ebca05fe91b3475919b56f06ac"
Jul 24 22:58:53.645473 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.645455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7" event={"ID":"90698fdb-a1f9-45ac-99f5-3a08112ac645","Type":"ContainerDied","Data":"19321907ea9f98f1fd5556fe0e2830c0dd57b54ef381acaa170fd3ee550a98cc"}
Jul 24 22:58:53.645556 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.645477    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7"
Jul 24 22:58:53.645556 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:53.645485    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19321907ea9f98f1fd5556fe0e2830c0dd57b54ef381acaa170fd3ee550a98cc"
Jul 24 22:58:54.972649 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:54.972610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:58.825057 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:58.825019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:58:59.954288 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954261    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"]
Jul 24 22:58:59.954779 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954759    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="util"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954781    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="util"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954793    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="util"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954800    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="util"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954813    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="util"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954821    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="util"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954830    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="extract"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954839    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="extract"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954856    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="pull"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954864    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="pull"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954876    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="pull"
Jul 24 22:58:59.954854 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954883    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="pull"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954892    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954899    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954906    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="pull"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954914    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="pull"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954926    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954933    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954941    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="pull"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954948    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="pull"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954970    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954978    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954988    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="util"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.954995    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="util"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.955097    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.955112    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb24c70-ab49-4285-b359-c278085e63e4" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.955154    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="02959104-3c14-4720-b78e-040bea03526d" containerName="extract"
Jul 24 22:58:59.955489 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.955164    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="90698fdb-a1f9-45ac-99f5-3a08112ac645" containerName="extract"
Jul 24 22:58:59.960622 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.960603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:58:59.962959 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.962938    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-grhwh\""
Jul 24 22:58:59.970819 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:58:59.970797    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"]
Jul 24 22:59:00.127082 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.127052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3753c6a6-75df-45c6-8e02-bec836052f52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.127261 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.127109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dblnj\" (UniqueName: \"kubernetes.io/projected/3753c6a6-75df-45c6-8e02-bec836052f52-kube-api-access-dblnj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.227646 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.227571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3753c6a6-75df-45c6-8e02-bec836052f52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.227646 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.227623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dblnj\" (UniqueName: \"kubernetes.io/projected/3753c6a6-75df-45c6-8e02-bec836052f52-kube-api-access-dblnj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.227978 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.227955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3753c6a6-75df-45c6-8e02-bec836052f52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.236356 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.236332    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:00.247236 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.247219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dblnj\" (UniqueName: \"kubernetes.io/projected/3753c6a6-75df-45c6-8e02-bec836052f52-kube-api-access-dblnj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.272102 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.272085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:00.428396 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.428372    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"]
Jul 24 22:59:00.430357 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:00.430334    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3753c6a6_75df_45c6_8e02_bec836052f52.slice/crio-b64ab54c74ba3cb7864031a34c933e5ce835ff3215e10dd29405c24c698f21f9 WatchSource:0}: Error finding container b64ab54c74ba3cb7864031a34c933e5ce835ff3215e10dd29405c24c698f21f9: Status 404 returned error can't find the container with id b64ab54c74ba3cb7864031a34c933e5ce835ff3215e10dd29405c24c698f21f9
Jul 24 22:59:00.605752 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.605724    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:00.683024 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:00.682993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt" event={"ID":"3753c6a6-75df-45c6-8e02-bec836052f52","Type":"ContainerStarted","Data":"b64ab54c74ba3cb7864031a34c933e5ce835ff3215e10dd29405c24c698f21f9"}
Jul 24 22:59:01.651620 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:01.651584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:02.684576 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.684543    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"]
Jul 24 22:59:02.688696 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.688668    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:02.691293 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.691267    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5lgx5\""
Jul 24 22:59:02.698889 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.698291    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"]
Jul 24 22:59:02.855618 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.855585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pfd5\" (UniqueName: \"kubernetes.io/projected/a4312f4f-93df-4d70-91fc-bec77b2ba638-kube-api-access-6pfd5\") pod \"authorino-operator-75cc8d5cb7-6vn45\" (UID: \"a4312f4f-93df-4d70-91fc-bec77b2ba638\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:02.956795 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.956715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pfd5\" (UniqueName: \"kubernetes.io/projected/a4312f4f-93df-4d70-91fc-bec77b2ba638-kube-api-access-6pfd5\") pod \"authorino-operator-75cc8d5cb7-6vn45\" (UID: \"a4312f4f-93df-4d70-91fc-bec77b2ba638\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:02.981399 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:02.981365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pfd5\" (UniqueName: \"kubernetes.io/projected/a4312f4f-93df-4d70-91fc-bec77b2ba638-kube-api-access-6pfd5\") pod \"authorino-operator-75cc8d5cb7-6vn45\" (UID: \"a4312f4f-93df-4d70-91fc-bec77b2ba638\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:03.003916 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:03.003892    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:03.146962 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:03.146926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"]
Jul 24 22:59:03.150749 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:03.150298    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4312f4f_93df_4d70_91fc_bec77b2ba638.slice/crio-477debc0d29a731883f9cedbf22a6fbc152e16def2a3242780bbeb1024c069f6 WatchSource:0}: Error finding container 477debc0d29a731883f9cedbf22a6fbc152e16def2a3242780bbeb1024c069f6: Status 404 returned error can't find the container with id 477debc0d29a731883f9cedbf22a6fbc152e16def2a3242780bbeb1024c069f6
Jul 24 22:59:03.189457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:03.189422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:03.290999 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:03.290893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:03.703609 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:03.703564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45" event={"ID":"a4312f4f-93df-4d70-91fc-bec77b2ba638","Type":"ContainerStarted","Data":"477debc0d29a731883f9cedbf22a6fbc152e16def2a3242780bbeb1024c069f6"}
Jul 24 22:59:04.713143 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.713035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt" event={"ID":"3753c6a6-75df-45c6-8e02-bec836052f52","Type":"ContainerStarted","Data":"cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36"}
Jul 24 22:59:04.713143 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.713104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:04.737937 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.737875    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt" podStartSLOduration=1.7725606329999999 podStartE2EDuration="5.737854842s" totalImagesPullingTime="3.965294209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:04.734970888 +0000 UTC m=+405.588297426" watchObservedRunningTime="2026-07-24 22:59:04.737854842 +0000 UTC m=+405.591181383"
Jul 24 22:59:04.799733 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.799693    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"]
Jul 24 22:59:04.807392 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.807368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:04.811723 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.811272    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5bp5n\""
Jul 24 22:59:04.811723 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.811551    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 22:59:04.815541 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.815514    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"]
Jul 24 22:59:04.977624 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:04.977538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5kt5\" (UniqueName: \"kubernetes.io/projected/18353da9-5b95-4d28-acce-c31238b94625-kube-api-access-t5kt5\") pod \"dns-operator-controller-manager-864dd78c4c-dzbwf\" (UID: \"18353da9-5b95-4d28-acce-c31238b94625\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:05.078828 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.078787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5kt5\" (UniqueName: \"kubernetes.io/projected/18353da9-5b95-4d28-acce-c31238b94625-kube-api-access-t5kt5\") pod \"dns-operator-controller-manager-864dd78c4c-dzbwf\" (UID: \"18353da9-5b95-4d28-acce-c31238b94625\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:05.088789 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.088763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5kt5\" (UniqueName: \"kubernetes.io/projected/18353da9-5b95-4d28-acce-c31238b94625-kube-api-access-t5kt5\") pod \"dns-operator-controller-manager-864dd78c4c-dzbwf\" (UID: \"18353da9-5b95-4d28-acce-c31238b94625\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:05.126783 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.126756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:05.145356 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.145330    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5658d96d79-vw77c"]
Jul 24 22:59:05.151752 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.151366    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.163067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.163034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5658d96d79-vw77c"]
Jul 24 22:59:05.208191 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.208160    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:05.280780 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.280752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-oauth-serving-cert\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.280925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.280794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-oauth-config\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.280925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.280870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-trusted-ca-bundle\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.280925 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.280908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-config\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.281067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.281001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-service-ca\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.281067 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.281033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8hlp\" (UniqueName: \"kubernetes.io/projected/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-kube-api-access-p8hlp\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.281154 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.281076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-serving-cert\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.299791 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.299756    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"]
Jul 24 22:59:05.301462 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.301425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:05.382301 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-service-ca\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8hlp\" (UniqueName: \"kubernetes.io/projected/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-kube-api-access-p8hlp\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-serving-cert\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-oauth-serving-cert\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382457 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-oauth-config\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382634 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-trusted-ca-bundle\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382634 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-config\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.382995 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.382969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-service-ca\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.383182 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.383162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-config\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.383448 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.383408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-oauth-serving-cert\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.383933 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.383909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-trusted-ca-bundle\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.386373 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.386341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-serving-cert\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.386480 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.386441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-console-oauth-config\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.391511 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.391486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:05.394101 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.394068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8hlp\" (UniqueName: \"kubernetes.io/projected/a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a-kube-api-access-p8hlp\") pod \"console-5658d96d79-vw77c\" (UID: \"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a\") " pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.478536 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.478496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:05.487430 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.487395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:05.946443 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:05.946413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:05.947070 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:05.947037    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18353da9_5b95_4d28_acce_c31238b94625.slice/crio-797572efd71620ebfb85308dcf7664c81d656899ad0847d9a3c91eb56c4dd617 WatchSource:0}: Error finding container 797572efd71620ebfb85308dcf7664c81d656899ad0847d9a3c91eb56c4dd617: Status 404 returned error can't find the container with id 797572efd71620ebfb85308dcf7664c81d656899ad0847d9a3c91eb56c4dd617
Jul 24 22:59:06.335719 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.335694    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5658d96d79-vw77c"]
Jul 24 22:59:06.336584 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:06.336559    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d1f2a5_3a0e_4d63_9df6_da62a6ae675a.slice/crio-2708e4a93aad28a03f5e3fb44a0630c329919ace0ca5c212bdfed82cc66e2bca WatchSource:0}: Error finding container 2708e4a93aad28a03f5e3fb44a0630c329919ace0ca5c212bdfed82cc66e2bca: Status 404 returned error can't find the container with id 2708e4a93aad28a03f5e3fb44a0630c329919ace0ca5c212bdfed82cc66e2bca
Jul 24 22:59:06.728729 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.728688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5658d96d79-vw77c" event={"ID":"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a","Type":"ContainerStarted","Data":"fceb3616f40386eb93fddca04bed688215d746974d516b834ba1e0955d4bdb47"}
Jul 24 22:59:06.728910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.728745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5658d96d79-vw77c" event={"ID":"a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a","Type":"ContainerStarted","Data":"2708e4a93aad28a03f5e3fb44a0630c329919ace0ca5c212bdfed82cc66e2bca"}
Jul 24 22:59:06.730118 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.730093    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf" event={"ID":"18353da9-5b95-4d28-acce-c31238b94625","Type":"ContainerStarted","Data":"797572efd71620ebfb85308dcf7664c81d656899ad0847d9a3c91eb56c4dd617"}
Jul 24 22:59:06.732007 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.731976    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45" event={"ID":"a4312f4f-93df-4d70-91fc-bec77b2ba638","Type":"ContainerStarted","Data":"651376e1ddb9bb4cb85d9ffd764e6d35234f5736f933111fcec39e0b93282a80"}
Jul 24 22:59:06.732201 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.732181    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:06.755154 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.755088    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5658d96d79-vw77c" podStartSLOduration=1.7550720640000002 podStartE2EDuration="1.755072064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:06.752092139 +0000 UTC m=+407.605418677" watchObservedRunningTime="2026-07-24 22:59:06.755072064 +0000 UTC m=+407.608398605"
Jul 24 22:59:06.772802 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.772752    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45" podStartSLOduration=1.666723401 podStartE2EDuration="4.772738944s" totalImagesPullingTime="3.106015543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:06.769470166 +0000 UTC m=+407.622796706" watchObservedRunningTime="2026-07-24 22:59:06.772738944 +0000 UTC m=+407.626065484"
Jul 24 22:59:06.795565 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.795543    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"]
Jul 24 22:59:06.800382 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.800364    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:06.802974 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.802954    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lmh85\""
Jul 24 22:59:06.815617 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.815594    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"]
Jul 24 22:59:06.895083 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.895051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h88df\" (UniqueName: \"kubernetes.io/projected/9ef7ec24-8736-4727-ac11-9a67fdb22a36-kube-api-access-h88df\") pod \"limitador-operator-controller-manager-859d7b78bd-49xkz\" (UID: \"9ef7ec24-8736-4727-ac11-9a67fdb22a36\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:06.995643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:06.995557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h88df\" (UniqueName: \"kubernetes.io/projected/9ef7ec24-8736-4727-ac11-9a67fdb22a36-kube-api-access-h88df\") pod \"limitador-operator-controller-manager-859d7b78bd-49xkz\" (UID: \"9ef7ec24-8736-4727-ac11-9a67fdb22a36\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:07.006802 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.006778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h88df\" (UniqueName: \"kubernetes.io/projected/9ef7ec24-8736-4727-ac11-9a67fdb22a36-kube-api-access-h88df\") pod \"limitador-operator-controller-manager-859d7b78bd-49xkz\" (UID: \"9ef7ec24-8736-4727-ac11-9a67fdb22a36\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:07.114787 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.114758    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:07.222574 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.222541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:07.304956 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.304926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"]
Jul 24 22:59:07.550917 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.550880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:07.556159 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:07.556076    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef7ec24_8736_4727_ac11_9a67fdb22a36.slice/crio-4f7e1ec93fba722626073ca5bfecb0aa977d69e4ea599000d579e6a896dd9c23 WatchSource:0}: Error finding container 4f7e1ec93fba722626073ca5bfecb0aa977d69e4ea599000d579e6a896dd9c23: Status 404 returned error can't find the container with id 4f7e1ec93fba722626073ca5bfecb0aa977d69e4ea599000d579e6a896dd9c23
Jul 24 22:59:07.646830 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.646785    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:07.739055 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.739018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf" event={"ID":"18353da9-5b95-4d28-acce-c31238b94625","Type":"ContainerStarted","Data":"2d858d381700bac92080cb8dde6b9f774aa8e0a132ef1b69c415ce33d64a5519"}
Jul 24 22:59:07.739237 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.739216    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:07.740686 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.740664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz" event={"ID":"9ef7ec24-8736-4727-ac11-9a67fdb22a36","Type":"ContainerStarted","Data":"4f7e1ec93fba722626073ca5bfecb0aa977d69e4ea599000d579e6a896dd9c23"}
Jul 24 22:59:07.764065 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:07.763976    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf" podStartSLOduration=2.104558004 podStartE2EDuration="3.763957553s" totalImagesPullingTime="1.659399549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:07.763020749 +0000 UTC m=+408.616347290" watchObservedRunningTime="2026-07-24 22:59:07.763957553 +0000 UTC m=+408.617284092"
Jul 24 22:59:08.979088 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:08.979056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:09.080680 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:09.080646    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:09.754207 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:09.754158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz" event={"ID":"9ef7ec24-8736-4727-ac11-9a67fdb22a36","Type":"ContainerStarted","Data":"f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065"}
Jul 24 22:59:09.754405 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:09.754236    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:09.772432 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:09.772360    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz" podStartSLOduration=2.39251935 podStartE2EDuration="3.772345625s" totalImagesPullingTime="1.379826275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:09.770914133 +0000 UTC m=+410.624240671" watchObservedRunningTime="2026-07-24 22:59:09.772345625 +0000 UTC m=+410.625672163"
Jul 24 22:59:14.739583 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:14.739548    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:14.800853 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:14.800822    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:15.479690 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.479659    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:15.479883 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.479714    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:15.484571 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.484552    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:15.721755 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.721725    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:15.790916 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.790840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5658d96d79-vw77c"
Jul 24 22:59:15.843138 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.843106    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbf9947f5-t69wd"]
Jul 24 22:59:15.954719 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:15.954687    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:16.056892 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:16.056825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:16.154214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:16.154167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:17.509507 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.509471    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"]
Jul 24 22:59:17.509884 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.509650    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" containerName="manager" containerID="cri-o://cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36" gracePeriod=2
Jul 24 22:59:17.525043 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.525000    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3753c6a6-75df-45c6-8e02-bec836052f52\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-fchtt\": pods \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:17.528511 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.528485    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"]
Jul 24 22:59:17.536804 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.536777    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"]
Jul 24 22:59:17.537085 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.537061    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" containerName="manager" containerID="cri-o://f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065" gracePeriod=2
Jul 24 22:59:17.539260 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.539242    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:17.539346 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.539306    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:17.540384 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.540359    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"]
Jul 24 22:59:17.540927 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.540912    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" containerName="manager"
Jul 24 22:59:17.541001 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.540932    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" containerName="manager"
Jul 24 22:59:17.541070 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.541059    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" containerName="manager"
Jul 24 22:59:17.544655 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.544637    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"]
Jul 24 22:59:17.544795 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.544726    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.554352 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.554316    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:17.563334 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.563310    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"]
Jul 24 22:59:17.567316 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.567297    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"]
Jul 24 22:59:17.567643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.567632    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" containerName="manager"
Jul 24 22:59:17.567694 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.567645    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" containerName="manager"
Jul 24 22:59:17.567725 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.567713    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" containerName="manager"
Jul 24 22:59:17.570595 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.570578    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:17.583887 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.583865    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"]
Jul 24 22:59:17.587269 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.587240    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:17.589166 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.589109    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fchtt\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:17.589748 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.589725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8dcd0c2-0820-49a4-a153-210d52b55140-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-svq9v\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.589832 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.589812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76d95\" (UniqueName: \"kubernetes.io/projected/c8dcd0c2-0820-49a4-a153-210d52b55140-kube-api-access-76d95\") pod \"kuadrant-operator-controller-manager-8684c9ff49-svq9v\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.590734 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.590713    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:17.690913 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.690877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8dcd0c2-0820-49a4-a153-210d52b55140-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-svq9v\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.691076 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.690958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76d95\" (UniqueName: \"kubernetes.io/projected/c8dcd0c2-0820-49a4-a153-210d52b55140-kube-api-access-76d95\") pod \"kuadrant-operator-controller-manager-8684c9ff49-svq9v\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.691076 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.691016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g56pn\" (UniqueName: \"kubernetes.io/projected/4fe88e94-84bd-4bba-b242-c19cbb0d6467-kube-api-access-g56pn\") pod \"limitador-operator-controller-manager-859d7b78bd-kq62x\" (UID: \"4fe88e94-84bd-4bba-b242-c19cbb0d6467\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:17.691335 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.691305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8dcd0c2-0820-49a4-a153-210d52b55140-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-svq9v\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.702051 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.702027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76d95\" (UniqueName: \"kubernetes.io/projected/c8dcd0c2-0820-49a4-a153-210d52b55140-kube-api-access-76d95\") pod \"kuadrant-operator-controller-manager-8684c9ff49-svq9v\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.742727 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.742700    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6vn45"
Jul 24 22:59:17.788029 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.788009    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:17.792307 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.792282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g56pn\" (UniqueName: \"kubernetes.io/projected/4fe88e94-84bd-4bba-b242-c19cbb0d6467-kube-api-access-g56pn\") pod \"limitador-operator-controller-manager-859d7b78bd-kq62x\" (UID: \"4fe88e94-84bd-4bba-b242-c19cbb0d6467\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:17.795640 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.795621    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:17.799948 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.799920    2564 generic.go:356] "Generic (PLEG): container finished" podID="3753c6a6-75df-45c6-8e02-bec836052f52" containerID="cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36" exitCode=0
Jul 24 22:59:17.800048 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.799957    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fchtt"
Jul 24 22:59:17.800048 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.799967    2564 scope.go:117] "RemoveContainer" containerID="cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36"
Jul 24 22:59:17.800871 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.800852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g56pn\" (UniqueName: \"kubernetes.io/projected/4fe88e94-84bd-4bba-b242-c19cbb0d6467-kube-api-access-g56pn\") pod \"limitador-operator-controller-manager-859d7b78bd-kq62x\" (UID: \"4fe88e94-84bd-4bba-b242-c19cbb0d6467\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:17.801614 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.801591    2564 generic.go:356] "Generic (PLEG): container finished" podID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" containerID="f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065" exitCode=0
Jul 24 22:59:17.801700 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.801631    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:17.810302 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.810286    2564 scope.go:117] "RemoveContainer" containerID="cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36"
Jul 24 22:59:17.810594 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.810563    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36\": container with ID starting with cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36 not found: ID does not exist" containerID="cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36"
Jul 24 22:59:17.810661 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.810598    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36"} err="failed to get container status \"cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36\": rpc error: code = NotFound desc = could not find container \"cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36\": container with ID starting with cac0d459862c6a9dc5658732203333c14eb88585de409e01be02476d539a1a36 not found: ID does not exist"
Jul 24 22:59:17.810661 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.810620    2564 scope.go:117] "RemoveContainer" containerID="f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065"
Jul 24 22:59:17.820545 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.820529    2564 scope.go:117] "RemoveContainer" containerID="f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065"
Jul 24 22:59:17.820792 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:17.820772    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065\": container with ID starting with f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065 not found: ID does not exist" containerID="f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065"
Jul 24 22:59:17.820838 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.820802    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065"} err="failed to get container status \"f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065\": rpc error: code = NotFound desc = could not find container \"f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065\": container with ID starting with f70d25736fe6c12776de3c3b823d169cd17254ce71a2b629777b0441a7e19065 not found: ID does not exist"
Jul 24 22:59:17.893315 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.893278    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3753c6a6-75df-45c6-8e02-bec836052f52-extensions-socket-volume\") pod \"3753c6a6-75df-45c6-8e02-bec836052f52\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") "
Jul 24 22:59:17.893455 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.893374    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dblnj\" (UniqueName: \"kubernetes.io/projected/3753c6a6-75df-45c6-8e02-bec836052f52-kube-api-access-dblnj\") pod \"3753c6a6-75df-45c6-8e02-bec836052f52\" (UID: \"3753c6a6-75df-45c6-8e02-bec836052f52\") "
Jul 24 22:59:17.893455 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.893450    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h88df\" (UniqueName: \"kubernetes.io/projected/9ef7ec24-8736-4727-ac11-9a67fdb22a36-kube-api-access-h88df\") pod \"9ef7ec24-8736-4727-ac11-9a67fdb22a36\" (UID: \"9ef7ec24-8736-4727-ac11-9a67fdb22a36\") "
Jul 24 22:59:17.893896 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.893871    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3753c6a6-75df-45c6-8e02-bec836052f52-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3753c6a6-75df-45c6-8e02-bec836052f52" (UID: "3753c6a6-75df-45c6-8e02-bec836052f52"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:59:17.895442 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.895420    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3753c6a6-75df-45c6-8e02-bec836052f52-kube-api-access-dblnj" (OuterVolumeSpecName: "kube-api-access-dblnj") pod "3753c6a6-75df-45c6-8e02-bec836052f52" (UID: "3753c6a6-75df-45c6-8e02-bec836052f52"). InnerVolumeSpecName "kube-api-access-dblnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:59:17.895759 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.895741    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef7ec24-8736-4727-ac11-9a67fdb22a36-kube-api-access-h88df" (OuterVolumeSpecName: "kube-api-access-h88df") pod "9ef7ec24-8736-4727-ac11-9a67fdb22a36" (UID: "9ef7ec24-8736-4727-ac11-9a67fdb22a36"). InnerVolumeSpecName "kube-api-access-h88df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:59:17.960544 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.960518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:17.967669 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.967565    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:17.994670 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.994641    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dblnj\" (UniqueName: \"kubernetes.io/projected/3753c6a6-75df-45c6-8e02-bec836052f52-kube-api-access-dblnj\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:17.994670 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.994671    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h88df\" (UniqueName: \"kubernetes.io/projected/9ef7ec24-8736-4727-ac11-9a67fdb22a36-kube-api-access-h88df\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:17.994806 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:17.994685    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3753c6a6-75df-45c6-8e02-bec836052f52-extensions-socket-volume\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:18.114244 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.114216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"]
Jul 24 22:59:18.115555 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:18.115519    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:18.116329 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:18.116307    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8dcd0c2_0820_49a4_a153_210d52b55140.slice/crio-5b626b0daf25f94f3303df7d877687799bc8d878bdb3786fc4398462af5507e8 WatchSource:0}: Error finding container 5b626b0daf25f94f3303df7d877687799bc8d878bdb3786fc4398462af5507e8: Status 404 returned error can't find the container with id 5b626b0daf25f94f3303df7d877687799bc8d878bdb3786fc4398462af5507e8
Jul 24 22:59:18.139642 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.139619    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"]
Jul 24 22:59:18.140385 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:18.140357    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe88e94_84bd_4bba_b242_c19cbb0d6467.slice/crio-69fe82feee91b0a0d7e3c42df41a240d6fa43bbdf61a5b83f6e8c3a0a00b9fe0 WatchSource:0}: Error finding container 69fe82feee91b0a0d7e3c42df41a240d6fa43bbdf61a5b83f6e8c3a0a00b9fe0: Status 404 returned error can't find the container with id 69fe82feee91b0a0d7e3c42df41a240d6fa43bbdf61a5b83f6e8c3a0a00b9fe0
Jul 24 22:59:18.269700 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.269613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:18.368762 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.368728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:18.750512 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.750485    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dzbwf"
Jul 24 22:59:18.774463 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:18.774435    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:18.808643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.808559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v" event={"ID":"c8dcd0c2-0820-49a4-a153-210d52b55140","Type":"ContainerStarted","Data":"06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa"}
Jul 24 22:59:18.808643 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.808613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v" event={"ID":"c8dcd0c2-0820-49a4-a153-210d52b55140","Type":"ContainerStarted","Data":"5b626b0daf25f94f3303df7d877687799bc8d878bdb3786fc4398462af5507e8"}
Jul 24 22:59:18.808857 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.808725    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:18.811373 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.811351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x" event={"ID":"4fe88e94-84bd-4bba-b242-c19cbb0d6467","Type":"ContainerStarted","Data":"20d15c9c77f750b2930be81ec96ff9746931de195860113615c9213c3e1fa436"}
Jul 24 22:59:18.811508 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.811392    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x" event={"ID":"4fe88e94-84bd-4bba-b242-c19cbb0d6467","Type":"ContainerStarted","Data":"69fe82feee91b0a0d7e3c42df41a240d6fa43bbdf61a5b83f6e8c3a0a00b9fe0"}
Jul 24 22:59:18.811508 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.811421    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:18.832297 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:18.832263    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:18.833387 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.833337    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v" podStartSLOduration=1.833324816 podStartE2EDuration="1.833324816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:18.830490153 +0000 UTC m=+419.683816693" watchObservedRunningTime="2026-07-24 22:59:18.833324816 +0000 UTC m=+419.686651356"
Jul 24 22:59:18.852768 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:18.852726    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x" podStartSLOduration=1.852712515 podStartE2EDuration="1.852712515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:18.851866065 +0000 UTC m=+419.705192603" watchObservedRunningTime="2026-07-24 22:59:18.852712515 +0000 UTC m=+419.706039053"
Jul 24 22:59:18.853771 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:18.853745    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:19.692599 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:19.692557    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-49xkz\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-49xkz"
Jul 24 22:59:19.694632 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:19.694609    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3753c6a6-75df-45c6-8e02-bec836052f52" path="/var/lib/kubelet/pods/3753c6a6-75df-45c6-8e02-bec836052f52/volumes"
Jul 24 22:59:19.694920 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:19.694907    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef7ec24-8736-4727-ac11-9a67fdb22a36" path="/var/lib/kubelet/pods/9ef7ec24-8736-4727-ac11-9a67fdb22a36/volumes"
Jul 24 22:59:20.652195 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:20.652157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:20.706620 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:20.706591    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:25.395586 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:25.395552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:25.499108 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:25.499066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:29.194691 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:29.194661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:29.295260 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:29.295223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:29.820791 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:29.820764    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:29.820963 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:29.820833    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kq62x"
Jul 24 22:59:37.833905 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:37.833865    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:37.933110 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:37.933066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:40.274456 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:40.274413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:40.336997 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:40.336944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:40.864603 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:40.864568    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dbf9947f5-t69wd" podUID="dda34048-c829-4281-a3a1-178c440038a9" containerName="console" containerID="cri-o://93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1" gracePeriod=15
Jul 24 22:59:41.122168 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.122092    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:59:41.191087 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191057    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-trusted-ca-bundle\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191087 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191085    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-console-config\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191112    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx2d5\" (UniqueName: \"kubernetes.io/projected/dda34048-c829-4281-a3a1-178c440038a9-kube-api-access-lx2d5\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191190    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-oauth-serving-cert\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191229    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-serving-cert\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191299 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191275    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-service-ca\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191507 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191311    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-oauth-config\") pod \"dda34048-c829-4281-a3a1-178c440038a9\" (UID: \"dda34048-c829-4281-a3a1-178c440038a9\") "
Jul 24 22:59:41.191507 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191470    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-console-config" (OuterVolumeSpecName: "console-config") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:59:41.191702 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191609    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:59:41.191702 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191684    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:59:41.191841 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.191785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-service-ca" (OuterVolumeSpecName: "service-ca") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:59:41.193443 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.193417    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:59:41.193549 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.193454    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:59:41.193594 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.193544    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda34048-c829-4281-a3a1-178c440038a9-kube-api-access-lx2d5" (OuterVolumeSpecName: "kube-api-access-lx2d5") pod "dda34048-c829-4281-a3a1-178c440038a9" (UID: "dda34048-c829-4281-a3a1-178c440038a9"). InnerVolumeSpecName "kube-api-access-lx2d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292118    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-oauth-serving-cert\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292171    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-serving-cert\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292181    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292189    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dda34048-c829-4281-a3a1-178c440038a9-console-oauth-config\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292197    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-trusted-ca-bundle\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292206    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dda34048-c829-4281-a3a1-178c440038a9-console-config\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.292214 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.292217    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx2d5\" (UniqueName: \"kubernetes.io/projected/dda34048-c829-4281-a3a1-178c440038a9-kube-api-access-lx2d5\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:41.955646 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.955557    2564 generic.go:356] "Generic (PLEG): container finished" podID="dda34048-c829-4281-a3a1-178c440038a9" containerID="93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1" exitCode=0
Jul 24 22:59:41.955646 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.955605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf9947f5-t69wd" event={"ID":"dda34048-c829-4281-a3a1-178c440038a9","Type":"ContainerDied","Data":"93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1"}
Jul 24 22:59:41.955646 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.955618    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf9947f5-t69wd"
Jul 24 22:59:41.956177 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.955680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf9947f5-t69wd" event={"ID":"dda34048-c829-4281-a3a1-178c440038a9","Type":"ContainerDied","Data":"747bdaa334840689775c8265f7dc6d95d23a4a4f857a6925b2ccdf5eab0311ec"}
Jul 24 22:59:41.956177 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.955706    2564 scope.go:117] "RemoveContainer" containerID="93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1"
Jul 24 22:59:41.964991 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.964956    2564 scope.go:117] "RemoveContainer" containerID="93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1"
Jul 24 22:59:41.965274 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:41.965258    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1\": container with ID starting with 93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1 not found: ID does not exist" containerID="93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1"
Jul 24 22:59:41.965356 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.965280    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1"} err="failed to get container status \"93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1\": rpc error: code = NotFound desc = could not find container \"93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1\": container with ID starting with 93a66907c5bc0aab6aea801021a5907a0427e13a5340bcb416482cf7163ca2d1 not found: ID does not exist"
Jul 24 22:59:41.973410 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.973389    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbf9947f5-t69wd"]
Jul 24 22:59:41.979309 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:41.979287    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dbf9947f5-t69wd"]
Jul 24 22:59:43.694563 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:43.694536    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda34048-c829-4281-a3a1-178c440038a9" path="/var/lib/kubelet/pods/dda34048-c829-4281-a3a1-178c440038a9/volumes"
Jul 24 22:59:44.374404 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.374372    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"]
Jul 24 22:59:44.374594 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.374569    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v" podUID="c8dcd0c2-0820-49a4-a153-210d52b55140" containerName="manager" containerID="cri-o://06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa" gracePeriod=10
Jul 24 22:59:44.622341 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.622317    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"]
Jul 24 22:59:44.622773 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.622760    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda34048-c829-4281-a3a1-178c440038a9" containerName="console"
Jul 24 22:59:44.622818 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.622775    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda34048-c829-4281-a3a1-178c440038a9" containerName="console"
Jul 24 22:59:44.622862 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.622853    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda34048-c829-4281-a3a1-178c440038a9" containerName="console"
Jul 24 22:59:44.625114 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.625071    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.634380 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.634363    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:44.641391 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.641371    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"]
Jul 24 22:59:44.722292 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.722267    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8dcd0c2-0820-49a4-a153-210d52b55140-extensions-socket-volume\") pod \"c8dcd0c2-0820-49a4-a153-210d52b55140\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") "
Jul 24 22:59:44.722727 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.722351    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76d95\" (UniqueName: \"kubernetes.io/projected/c8dcd0c2-0820-49a4-a153-210d52b55140-kube-api-access-76d95\") pod \"c8dcd0c2-0820-49a4-a153-210d52b55140\" (UID: \"c8dcd0c2-0820-49a4-a153-210d52b55140\") "
Jul 24 22:59:44.722727 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.722511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b64bce8-c92b-49a7-8359-9809c1e01178-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9xxw2\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.722727 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.722560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b78d\" (UniqueName: \"kubernetes.io/projected/5b64bce8-c92b-49a7-8359-9809c1e01178-kube-api-access-5b78d\") pod \"kuadrant-operator-controller-manager-86878c769f-9xxw2\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.722910 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.722755    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dcd0c2-0820-49a4-a153-210d52b55140-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8dcd0c2-0820-49a4-a153-210d52b55140" (UID: "c8dcd0c2-0820-49a4-a153-210d52b55140"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:59:44.724447 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.724419    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8dcd0c2-0820-49a4-a153-210d52b55140-kube-api-access-76d95" (OuterVolumeSpecName: "kube-api-access-76d95") pod "c8dcd0c2-0820-49a4-a153-210d52b55140" (UID: "c8dcd0c2-0820-49a4-a153-210d52b55140"). InnerVolumeSpecName "kube-api-access-76d95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:59:44.823249 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.823216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b64bce8-c92b-49a7-8359-9809c1e01178-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9xxw2\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.823385 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.823260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b78d\" (UniqueName: \"kubernetes.io/projected/5b64bce8-c92b-49a7-8359-9809c1e01178-kube-api-access-5b78d\") pod \"kuadrant-operator-controller-manager-86878c769f-9xxw2\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.823385 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.823317    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76d95\" (UniqueName: \"kubernetes.io/projected/c8dcd0c2-0820-49a4-a153-210d52b55140-kube-api-access-76d95\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:44.823385 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.823327    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8dcd0c2-0820-49a4-a153-210d52b55140-extensions-socket-volume\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 22:59:44.823621 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.823589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b64bce8-c92b-49a7-8359-9809c1e01178-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9xxw2\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.831572 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.831552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b78d\" (UniqueName: \"kubernetes.io/projected/5b64bce8-c92b-49a7-8359-9809c1e01178-kube-api-access-5b78d\") pod \"kuadrant-operator-controller-manager-86878c769f-9xxw2\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.942072 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.942010    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:44.974852 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.974826    2564 generic.go:356] "Generic (PLEG): container finished" podID="c8dcd0c2-0820-49a4-a153-210d52b55140" containerID="06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa" exitCode=0
Jul 24 22:59:44.974952 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.974868    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"
Jul 24 22:59:44.974952 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.974913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v" event={"ID":"c8dcd0c2-0820-49a4-a153-210d52b55140","Type":"ContainerDied","Data":"06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa"}
Jul 24 22:59:44.975030 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.974973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v" event={"ID":"c8dcd0c2-0820-49a4-a153-210d52b55140","Type":"ContainerDied","Data":"5b626b0daf25f94f3303df7d877687799bc8d878bdb3786fc4398462af5507e8"}
Jul 24 22:59:44.975030 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.975005    2564 scope.go:117] "RemoveContainer" containerID="06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa"
Jul 24 22:59:44.986038 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.986018    2564 scope.go:117] "RemoveContainer" containerID="06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa"
Jul 24 22:59:44.986364 ip-10-0-143-78 kubenswrapper[2564]: E0724 22:59:44.986340    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa\": container with ID starting with 06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa not found: ID does not exist" containerID="06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa"
Jul 24 22:59:44.986450 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:44.986376    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa"} err="failed to get container status \"06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa\": rpc error: code = NotFound desc = could not find container \"06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa\": container with ID starting with 06439be7247333e2e46b18ee6c60b463dd9ce904f73bdca12becfeef117003fa not found: ID does not exist"
Jul 24 22:59:45.007372 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.007327    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"]
Jul 24 22:59:45.017301 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.017278    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-svq9v"]
Jul 24 22:59:45.036498 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.036468    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:45.086940 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.086918    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"]
Jul 24 22:59:45.089864 ip-10-0-143-78 kubenswrapper[2564]: W0724 22:59:45.089837    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b64bce8_c92b_49a7_8359_9809c1e01178.slice/crio-b074da77ff76ae1939fc26020a381d4a119fcb2750806d3d215ed4ff854dbcaf WatchSource:0}: Error finding container b074da77ff76ae1939fc26020a381d4a119fcb2750806d3d215ed4ff854dbcaf: Status 404 returned error can't find the container with id b074da77ff76ae1939fc26020a381d4a119fcb2750806d3d215ed4ff854dbcaf
Jul 24 22:59:45.105659 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.105632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:45.694904 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.694876    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8dcd0c2-0820-49a4-a153-210d52b55140" path="/var/lib/kubelet/pods/c8dcd0c2-0820-49a4-a153-210d52b55140/volumes"
Jul 24 22:59:45.982945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.982873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2" event={"ID":"5b64bce8-c92b-49a7-8359-9809c1e01178","Type":"ContainerStarted","Data":"100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc"}
Jul 24 22:59:45.982945 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.982917    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2" event={"ID":"5b64bce8-c92b-49a7-8359-9809c1e01178","Type":"ContainerStarted","Data":"b074da77ff76ae1939fc26020a381d4a119fcb2750806d3d215ed4ff854dbcaf"}
Jul 24 22:59:45.983337 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:45.982975    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:46.006562 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:46.006520    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2" podStartSLOduration=2.006507273 podStartE2EDuration="2.006507273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:46.004847993 +0000 UTC m=+446.858174533" watchObservedRunningTime="2026-07-24 22:59:46.006507273 +0000 UTC m=+446.859833811"
Jul 24 22:59:49.227692 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:49.227655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:49.331267 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:49.331218    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:56.996607 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:56.996581    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 22:59:58.085359 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:58.085323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 22:59:58.177551 ip-10-0-143-78 kubenswrapper[2564]: I0724 22:59:58.177523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:00.479047 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:00.479008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:00.532325 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:00.532297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:00.852562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:00.852526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:01.049809 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:01.049772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:03.176446 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.176404    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:03.340319 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.340270    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:03.441298 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.441188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:03.507951 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.507918    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:00:03.508431 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.508403    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8dcd0c2-0820-49a4-a153-210d52b55140" containerName="manager"
Jul 24 23:00:03.508431 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.508426    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8dcd0c2-0820-49a4-a153-210d52b55140" containerName="manager"
Jul 24 23:00:03.508646 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.508581    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8dcd0c2-0820-49a4-a153-210d52b55140" containerName="manager"
Jul 24 23:00:03.513269 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.513252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.515428 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.515406    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 23:00:03.516420 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.516393    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6p7bk\""
Jul 24 23:00:03.519533 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.519512    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:00:03.551696 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.551670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:00:03.672138 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.672103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czrmm\" (UniqueName: \"kubernetes.io/projected/31fffaa6-f925-4523-bd3b-369d1fd3d6c3-kube-api-access-czrmm\") pod \"limitador-limitador-54d6d988d5-hgjfj\" (UID: \"31fffaa6-f925-4523-bd3b-369d1fd3d6c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.672290 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.672223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31fffaa6-f925-4523-bd3b-369d1fd3d6c3-config-file\") pod \"limitador-limitador-54d6d988d5-hgjfj\" (UID: \"31fffaa6-f925-4523-bd3b-369d1fd3d6c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.773286 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.773194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czrmm\" (UniqueName: \"kubernetes.io/projected/31fffaa6-f925-4523-bd3b-369d1fd3d6c3-kube-api-access-czrmm\") pod \"limitador-limitador-54d6d988d5-hgjfj\" (UID: \"31fffaa6-f925-4523-bd3b-369d1fd3d6c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.773286 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.773251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31fffaa6-f925-4523-bd3b-369d1fd3d6c3-config-file\") pod \"limitador-limitador-54d6d988d5-hgjfj\" (UID: \"31fffaa6-f925-4523-bd3b-369d1fd3d6c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.773862 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.773843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31fffaa6-f925-4523-bd3b-369d1fd3d6c3-config-file\") pod \"limitador-limitador-54d6d988d5-hgjfj\" (UID: \"31fffaa6-f925-4523-bd3b-369d1fd3d6c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.781080 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.781057    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czrmm\" (UniqueName: \"kubernetes.io/projected/31fffaa6-f925-4523-bd3b-369d1fd3d6c3-kube-api-access-czrmm\") pod \"limitador-limitador-54d6d988d5-hgjfj\" (UID: \"31fffaa6-f925-4523-bd3b-369d1fd3d6c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.825052 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.825029    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:03.956496 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:03.956472    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:00:03.958726 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:00:03.958699    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fffaa6_f925_4523_bd3b_369d1fd3d6c3.slice/crio-bc41d3639833cb9967b46a6e09b085c63f4fb891936d95d95b8e5a9b73d485e2 WatchSource:0}: Error finding container bc41d3639833cb9967b46a6e09b085c63f4fb891936d95d95b8e5a9b73d485e2: Status 404 returned error can't find the container with id bc41d3639833cb9967b46a6e09b085c63f4fb891936d95d95b8e5a9b73d485e2
Jul 24 23:00:04.027716 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.027646    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-96g4d"]
Jul 24 23:00:04.032194 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.032177    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:04.034459 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.034441    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lskhv\""
Jul 24 23:00:04.042842 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.042771    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-96g4d"]
Jul 24 23:00:04.052830 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.052802    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:04.087973 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.087950    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj" event={"ID":"31fffaa6-f925-4523-bd3b-369d1fd3d6c3","Type":"ContainerStarted","Data":"bc41d3639833cb9967b46a6e09b085c63f4fb891936d95d95b8e5a9b73d485e2"}
Jul 24 23:00:04.155996 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.155972    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:04.175788 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.175762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dksjm\" (UniqueName: \"kubernetes.io/projected/3a7d6714-6511-4e4d-9a49-c07063b89792-kube-api-access-dksjm\") pod \"authorino-58cd955675-96g4d\" (UID: \"3a7d6714-6511-4e4d-9a49-c07063b89792\") " pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:04.277262 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.277229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dksjm\" (UniqueName: \"kubernetes.io/projected/3a7d6714-6511-4e4d-9a49-c07063b89792-kube-api-access-dksjm\") pod \"authorino-58cd955675-96g4d\" (UID: \"3a7d6714-6511-4e4d-9a49-c07063b89792\") " pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:04.284533 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.284476    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dksjm\" (UniqueName: \"kubernetes.io/projected/3a7d6714-6511-4e4d-9a49-c07063b89792-kube-api-access-dksjm\") pod \"authorino-58cd955675-96g4d\" (UID: \"3a7d6714-6511-4e4d-9a49-c07063b89792\") " pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:04.343280 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.343260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:04.489659 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:04.489630    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-96g4d"]
Jul 24 23:00:04.492019 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:00:04.491980    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a7d6714_6511_4e4d_9a49_c07063b89792.slice/crio-ccfb2180c5a04c8a845b5f23648cf398279cc2fa25e29c1ca5011d36934d3d98 WatchSource:0}: Error finding container ccfb2180c5a04c8a845b5f23648cf398279cc2fa25e29c1ca5011d36934d3d98: Status 404 returned error can't find the container with id ccfb2180c5a04c8a845b5f23648cf398279cc2fa25e29c1ca5011d36934d3d98
Jul 24 23:00:05.097493 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:05.097455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-96g4d" event={"ID":"3a7d6714-6511-4e4d-9a49-c07063b89792","Type":"ContainerStarted","Data":"ccfb2180c5a04c8a845b5f23648cf398279cc2fa25e29c1ca5011d36934d3d98"}
Jul 24 23:00:05.271365 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:05.271283    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:05.375508 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:05.375393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:09.124864 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:09.124831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj" event={"ID":"31fffaa6-f925-4523-bd3b-369d1fd3d6c3","Type":"ContainerStarted","Data":"51b7e39202281ab51d11407ccedad0245b48b3030825b2b23191b9448e8a01ac"}
Jul 24 23:00:09.125379 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:09.124928    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:09.126440 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:09.126415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-96g4d" event={"ID":"3a7d6714-6511-4e4d-9a49-c07063b89792","Type":"ContainerStarted","Data":"96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85"}
Jul 24 23:00:09.141296 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:09.141255    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj" podStartSLOduration=1.635060953 podStartE2EDuration="6.141241481s" totalImagesPullingTime="4.506180528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:09.140410689 +0000 UTC m=+469.993737229" watchObservedRunningTime="2026-07-24 23:00:09.141241481 +0000 UTC m=+469.994568021"
Jul 24 23:00:09.155013 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:09.154976    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-96g4d" podStartSLOduration=1.1819040570000001 podStartE2EDuration="5.154962335s" totalImagesPullingTime="3.973058278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:09.153636435 +0000 UTC m=+470.006962975" watchObservedRunningTime="2026-07-24 23:00:09.154962335 +0000 UTC m=+470.008288873"
Jul 24 23:00:09.526769 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:09.526701    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-96g4d"]
Jul 24 23:00:11.138603 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:11.138561    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-96g4d" podUID="3a7d6714-6511-4e4d-9a49-c07063b89792" containerName="authorino" containerID="cri-o://96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85" gracePeriod=30
Jul 24 23:00:11.377657 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:11.377634    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:11.536932 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:11.536843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dksjm\" (UniqueName: \"kubernetes.io/projected/3a7d6714-6511-4e4d-9a49-c07063b89792-kube-api-access-dksjm\") pod \"3a7d6714-6511-4e4d-9a49-c07063b89792\" (UID: \"3a7d6714-6511-4e4d-9a49-c07063b89792\") "
Jul 24 23:00:11.538985 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:11.538954    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a7d6714-6511-4e4d-9a49-c07063b89792-kube-api-access-dksjm" (OuterVolumeSpecName: "kube-api-access-dksjm") pod "3a7d6714-6511-4e4d-9a49-c07063b89792" (UID: "3a7d6714-6511-4e4d-9a49-c07063b89792"). InnerVolumeSpecName "kube-api-access-dksjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:00:11.637898 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:11.637875    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dksjm\" (UniqueName: \"kubernetes.io/projected/3a7d6714-6511-4e4d-9a49-c07063b89792-kube-api-access-dksjm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:00:12.146018 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.145983    2564 generic.go:356] "Generic (PLEG): container finished" podID="3a7d6714-6511-4e4d-9a49-c07063b89792" containerID="96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85" exitCode=0
Jul 24 23:00:12.146018 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.146019    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-96g4d"
Jul 24 23:00:12.146537 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.146070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-96g4d" event={"ID":"3a7d6714-6511-4e4d-9a49-c07063b89792","Type":"ContainerDied","Data":"96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85"}
Jul 24 23:00:12.146537 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.146143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-96g4d" event={"ID":"3a7d6714-6511-4e4d-9a49-c07063b89792","Type":"ContainerDied","Data":"ccfb2180c5a04c8a845b5f23648cf398279cc2fa25e29c1ca5011d36934d3d98"}
Jul 24 23:00:12.146537 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.146173    2564 scope.go:117] "RemoveContainer" containerID="96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85"
Jul 24 23:00:12.158248 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.158227    2564 scope.go:117] "RemoveContainer" containerID="96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85"
Jul 24 23:00:12.158497 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:00:12.158481    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85\": container with ID starting with 96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85 not found: ID does not exist" containerID="96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85"
Jul 24 23:00:12.158552 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.158507    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85"} err="failed to get container status \"96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85\": rpc error: code = NotFound desc = could not find container \"96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85\": container with ID starting with 96662c6e7d84a48fc35085d8bb5c9a26de05bff63dc0167e8cf006d5ab706e85 not found: ID does not exist"
Jul 24 23:00:12.162028 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.162007    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-96g4d"]
Jul 24 23:00:12.165539 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:12.165519    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-96g4d"]
Jul 24 23:00:13.694783 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:13.694750    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a7d6714-6511-4e4d-9a49-c07063b89792" path="/var/lib/kubelet/pods/3a7d6714-6511-4e4d-9a49-c07063b89792/volumes"
Jul 24 23:00:20.133450 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:20.133422    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"
Jul 24 23:00:24.444609 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:24.444566    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:24.586606 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:24.586574    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:24.928913 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:24.928876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:30.520557 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:30.520516    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:44.959205 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:44.959175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:45.151569 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.151535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:45.178743 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.178711    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mxbrz"]
Jul 24 23:00:45.179319 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.179294    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a7d6714-6511-4e4d-9a49-c07063b89792" containerName="authorino"
Jul 24 23:00:45.179319 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.179317    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a7d6714-6511-4e4d-9a49-c07063b89792" containerName="authorino"
Jul 24 23:00:45.179481 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.179376    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a7d6714-6511-4e4d-9a49-c07063b89792" containerName="authorino"
Jul 24 23:00:45.181340 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.181322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:45.183805 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.183787    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lskhv\""
Jul 24 23:00:45.191586 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.191563    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mxbrz"]
Jul 24 23:00:45.306421 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.306338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r49w\" (UniqueName: \"kubernetes.io/projected/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0-kube-api-access-5r49w\") pod \"authorino-5d4b67c446-mxbrz\" (UID: \"41b26e93-8b4a-41af-a3d1-bb65fe32b9e0\") " pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:45.407844 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.407809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r49w\" (UniqueName: \"kubernetes.io/projected/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0-kube-api-access-5r49w\") pod \"authorino-5d4b67c446-mxbrz\" (UID: \"41b26e93-8b4a-41af-a3d1-bb65fe32b9e0\") " pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:45.415419 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.415391    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mxbrz"]
Jul 24 23:00:45.415633 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:00:45.415614    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5r49w], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-mxbrz" podUID="41b26e93-8b4a-41af-a3d1-bb65fe32b9e0"
Jul 24 23:00:45.417849 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:45.417817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r49w\" (UniqueName: \"kubernetes.io/projected/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0-kube-api-access-5r49w\") pod \"authorino-5d4b67c446-mxbrz\" (UID: \"41b26e93-8b4a-41af-a3d1-bb65fe32b9e0\") " pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:46.334931 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:46.334897    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:46.341154 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:46.341133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:46.416107 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:46.416084    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r49w\" (UniqueName: \"kubernetes.io/projected/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0-kube-api-access-5r49w\") pod \"41b26e93-8b4a-41af-a3d1-bb65fe32b9e0\" (UID: \"41b26e93-8b4a-41af-a3d1-bb65fe32b9e0\") "
Jul 24 23:00:46.418023 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:46.417994    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0-kube-api-access-5r49w" (OuterVolumeSpecName: "kube-api-access-5r49w") pod "41b26e93-8b4a-41af-a3d1-bb65fe32b9e0" (UID: "41b26e93-8b4a-41af-a3d1-bb65fe32b9e0"). InnerVolumeSpecName "kube-api-access-5r49w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:00:46.516670 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:46.516641    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r49w\" (UniqueName: \"kubernetes.io/projected/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0-kube-api-access-5r49w\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:00:47.341469 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:47.341438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mxbrz"
Jul 24 23:00:47.373236 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:47.373210    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mxbrz"]
Jul 24 23:00:47.382681 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:47.382657    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mxbrz"]
Jul 24 23:00:47.694887 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:47.694813    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41b26e93-8b4a-41af-a3d1-bb65fe32b9e0" path="/var/lib/kubelet/pods/41b26e93-8b4a-41af-a3d1-bb65fe32b9e0/volumes"
Jul 24 23:00:51.309294 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:51.309257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:51.431334 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:51.431300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:51.693027 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:51.692985    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:55.036160 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:55.036101    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:00:55.133714 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:00:55.133688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:12.797657 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:12.797575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:13.480772 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:13.480742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:27.886138 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:27.886078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:27.989849 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:27.989811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:28.519586 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:28.519551    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:28.742091 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:28.742054    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:28.995993 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:28.995957    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:29.086954 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.086924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:29.165874 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.165844    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"]
Jul 24 23:01:29.169455 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.169442    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.171788 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.171768    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pdxdc\""
Jul 24 23:01:29.171788 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.171781    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 23:01:29.180686 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.180666    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"]
Jul 24 23:01:29.205396 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.205372    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:01:29.284879 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.284808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdx2\" (UniqueName: \"kubernetes.io/projected/b244bf57-7ebf-4e84-8661-5ce003e1bfe3-kube-api-access-jpdx2\") pod \"payload-pre-processing-687dbb6cf6-kc9w5\" (UID: \"b244bf57-7ebf-4e84-8661-5ce003e1bfe3\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.285040 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.284933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b244bf57-7ebf-4e84-8661-5ce003e1bfe3-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-kc9w5\" (UID: \"b244bf57-7ebf-4e84-8661-5ce003e1bfe3\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.386251 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.386215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b244bf57-7ebf-4e84-8661-5ce003e1bfe3-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-kc9w5\" (UID: \"b244bf57-7ebf-4e84-8661-5ce003e1bfe3\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.386411 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.386297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdx2\" (UniqueName: \"kubernetes.io/projected/b244bf57-7ebf-4e84-8661-5ce003e1bfe3-kube-api-access-jpdx2\") pod \"payload-pre-processing-687dbb6cf6-kc9w5\" (UID: \"b244bf57-7ebf-4e84-8661-5ce003e1bfe3\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.386837 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.386820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b244bf57-7ebf-4e84-8661-5ce003e1bfe3-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-kc9w5\" (UID: \"b244bf57-7ebf-4e84-8661-5ce003e1bfe3\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.393592 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.393566    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdx2\" (UniqueName: \"kubernetes.io/projected/b244bf57-7ebf-4e84-8661-5ce003e1bfe3-kube-api-access-jpdx2\") pod \"payload-pre-processing-687dbb6cf6-kc9w5\" (UID: \"b244bf57-7ebf-4e84-8661-5ce003e1bfe3\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.482867 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.482837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:29.620614 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:29.620589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"]
Jul 24 23:01:29.622358 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:01:29.622328    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb244bf57_7ebf_4e84_8661_5ce003e1bfe3.slice/crio-a677ae0c2e658acb7d72ee2c066d51519df4ff00fe2b504e790d7b4371693511 WatchSource:0}: Error finding container a677ae0c2e658acb7d72ee2c066d51519df4ff00fe2b504e790d7b4371693511: Status 404 returned error can't find the container with id a677ae0c2e658acb7d72ee2c066d51519df4ff00fe2b504e790d7b4371693511
Jul 24 23:01:30.594089 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:30.594046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5" event={"ID":"b244bf57-7ebf-4e84-8661-5ce003e1bfe3","Type":"ContainerStarted","Data":"a677ae0c2e658acb7d72ee2c066d51519df4ff00fe2b504e790d7b4371693511"}
Jul 24 23:01:31.607775 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:31.607741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5" event={"ID":"b244bf57-7ebf-4e84-8661-5ce003e1bfe3","Type":"ContainerStarted","Data":"af91268ffbc33d0d35753f499cc6185b6b90dfa15edceb3c89d08261e1e4dbfb"}
Jul 24 23:01:31.608169 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:31.607813    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:01:31.623410 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:31.623368    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5" podStartSLOduration=1.155220442 podStartE2EDuration="2.623355762s" totalImagesPullingTime="1.46813532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:01:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:01:31.622030693 +0000 UTC m=+552.475357226" watchObservedRunningTime="2026-07-24 23:01:31.623355762 +0000 UTC m=+552.476682300"
Jul 24 23:01:39.184408 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:39.184355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:39.278108 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:39.278064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:42.615841 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:42.615801    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5" podUID="b244bf57-7ebf-4e84-8661-5ce003e1bfe3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.56:9004: connect: connection refused"
Jul 24 23:01:46.507070 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:46.507032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:01:47.089585 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:47.089547    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:01:49.484168 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:49.484111    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5" podUID="b244bf57-7ebf-4e84-8661-5ce003e1bfe3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.56:9004: connect: connection refused"
Jul 24 23:01:52.614662 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:01:52.614615    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5" podUID="b244bf57-7ebf-4e84-8661-5ce003e1bfe3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.56:9004: connect: connection refused"
Jul 24 23:02:02.615038 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:02.615005    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kc9w5"
Jul 24 23:02:04.223929 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:04.223888    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:04.992053 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:04.992016    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:02:05.174226 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:05.174188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:05.277664 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:05.277583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:11.592707 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:11.592669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:12.692521 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:12.692486    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:02:16.611627 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:16.611593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:18.986694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:18.986658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:02:19.566310 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:19.566280    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:02:19.567836 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:19.567813    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:02:20.453413 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:20.453370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:20.865094 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:20.865057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:20.966600 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:20.966553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:32.086288 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.086020    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"]
Jul 24 23:02:32.089896 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.089876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.092272 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.092251    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 23:02:32.093163 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.093143    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 24 23:02:32.093163 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.093151    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 23:02:32.093321 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.093191    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q7cqg\""
Jul 24 23:02:32.099046 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.099025    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"]
Jul 24 23:02:32.117079 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.117042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:32.228757 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.228730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.228913 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.228767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.228913 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.228790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.228913 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.228840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6gwn\" (UniqueName: \"kubernetes.io/projected/17af7937-9522-4680-967b-755f369a83f9-kube-api-access-d6gwn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.228913 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.228872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.229093 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.228975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17af7937-9522-4680-967b-755f369a83f9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330082 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6gwn\" (UniqueName: \"kubernetes.io/projected/17af7937-9522-4680-967b-755f369a83f9-kube-api-access-d6gwn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17af7937-9522-4680-967b-755f369a83f9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330536 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330597 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.330656 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.330631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.332324 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.332302    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17af7937-9522-4680-967b-755f369a83f9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.332640 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.332621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17af7937-9522-4680-967b-755f369a83f9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.337867 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.337813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6gwn\" (UniqueName: \"kubernetes.io/projected/17af7937-9522-4680-967b-755f369a83f9-kube-api-access-d6gwn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gr6dx\" (UID: \"17af7937-9522-4680-967b-755f369a83f9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.402472 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.402444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:32.541345 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.541316    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"]
Jul 24 23:02:32.542983 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:02:32.542954    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17af7937_9522_4680_967b_755f369a83f9.slice/crio-a919dc4f88b50ea202b8744986e15f47f7504a989b345b774955ed1b0b196154 WatchSource:0}: Error finding container a919dc4f88b50ea202b8744986e15f47f7504a989b345b774955ed1b0b196154: Status 404 returned error can't find the container with id a919dc4f88b50ea202b8744986e15f47f7504a989b345b774955ed1b0b196154
Jul 24 23:02:32.544681 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.544662    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:02:32.938958 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:32.938910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx" event={"ID":"17af7937-9522-4680-967b-755f369a83f9","Type":"ContainerStarted","Data":"a919dc4f88b50ea202b8744986e15f47f7504a989b345b774955ed1b0b196154"}
Jul 24 23:02:33.946682 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:33.946650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx" event={"ID":"17af7937-9522-4680-967b-755f369a83f9","Type":"ContainerStarted","Data":"aa077d950c7d56fc714f0b1c6c4b9cdcfba0617d0e7507f7be58d853d7d64263"}
Jul 24 23:02:33.962477 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:33.962427    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx" podStartSLOduration=0.964603487 podStartE2EDuration="1.962411358s" totalImagesPullingTime="997.807871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:33.961938824 +0000 UTC m=+614.815265367" watchObservedRunningTime="2026-07-24 23:02:33.962411358 +0000 UTC m=+614.815737897"
Jul 24 23:02:34.815589 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:34.815556    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:34.987933 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:34.987900    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:02:36.238220 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:36.238179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:02:42.403145 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:42.403033    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:42.403145 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:42.403101    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:42.415205 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:42.415182    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:43.008326 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:43.008298    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gr6dx"
Jul 24 23:02:46.700514 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:46.700477    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:02:56.643661 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:02:56.643611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:03:07.702384 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:07.702346    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:03:30.570816 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.570784    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5668767cc9-gqqnh"]
Jul 24 23:03:30.574937 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.574918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.578400 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.578380    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 23:03:30.578526 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.578490    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lskhv\""
Jul 24 23:03:30.584756 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.584734    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5668767cc9-gqqnh"]
Jul 24 23:03:30.616745 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.616719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ded3901-d129-4ac2-b2fb-998ee9b7b794-tls-cert\") pod \"authorino-5668767cc9-gqqnh\" (UID: \"8ded3901-d129-4ac2-b2fb-998ee9b7b794\") " pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.616891 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.616764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25glt\" (UniqueName: \"kubernetes.io/projected/8ded3901-d129-4ac2-b2fb-998ee9b7b794-kube-api-access-25glt\") pod \"authorino-5668767cc9-gqqnh\" (UID: \"8ded3901-d129-4ac2-b2fb-998ee9b7b794\") " pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.717534 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.717495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ded3901-d129-4ac2-b2fb-998ee9b7b794-tls-cert\") pod \"authorino-5668767cc9-gqqnh\" (UID: \"8ded3901-d129-4ac2-b2fb-998ee9b7b794\") " pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.717696 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.717549    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25glt\" (UniqueName: \"kubernetes.io/projected/8ded3901-d129-4ac2-b2fb-998ee9b7b794-kube-api-access-25glt\") pod \"authorino-5668767cc9-gqqnh\" (UID: \"8ded3901-d129-4ac2-b2fb-998ee9b7b794\") " pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.720044 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.720019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ded3901-d129-4ac2-b2fb-998ee9b7b794-tls-cert\") pod \"authorino-5668767cc9-gqqnh\" (UID: \"8ded3901-d129-4ac2-b2fb-998ee9b7b794\") " pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.730745 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.730720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25glt\" (UniqueName: \"kubernetes.io/projected/8ded3901-d129-4ac2-b2fb-998ee9b7b794-kube-api-access-25glt\") pod \"authorino-5668767cc9-gqqnh\" (UID: \"8ded3901-d129-4ac2-b2fb-998ee9b7b794\") " pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:30.884594 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:30.884561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5668767cc9-gqqnh"
Jul 24 23:03:31.042214 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:31.042191    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5668767cc9-gqqnh"]
Jul 24 23:03:31.044094 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:03:31.044067    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ded3901_d129_4ac2_b2fb_998ee9b7b794.slice/crio-6386a242fa39fe53948b51ebb1c28f959d7e5f1af342a60f930034e3e8e84317 WatchSource:0}: Error finding container 6386a242fa39fe53948b51ebb1c28f959d7e5f1af342a60f930034e3e8e84317: Status 404 returned error can't find the container with id 6386a242fa39fe53948b51ebb1c28f959d7e5f1af342a60f930034e3e8e84317
Jul 24 23:03:31.052908 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:31.052864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:03:31.147296 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:31.147182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:03:31.274102 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:31.274071    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5668767cc9-gqqnh" event={"ID":"8ded3901-d129-4ac2-b2fb-998ee9b7b794","Type":"ContainerStarted","Data":"6386a242fa39fe53948b51ebb1c28f959d7e5f1af342a60f930034e3e8e84317"}
Jul 24 23:03:32.206216 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:32.206186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:03:32.281629 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:32.281582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5668767cc9-gqqnh" event={"ID":"8ded3901-d129-4ac2-b2fb-998ee9b7b794","Type":"ContainerStarted","Data":"68174f0aef10d0ec248be4f596ad95305db8556189e0511d4bad4ef77cf82e9f"}
Jul 24 23:03:32.299728 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:32.299682    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5668767cc9-gqqnh" podStartSLOduration=1.8818314969999999 podStartE2EDuration="2.299665236s" totalImagesPullingTime="417.833739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:03:32.298631384 +0000 UTC m=+673.151957923" watchObservedRunningTime="2026-07-24 23:03:32.299665236 +0000 UTC m=+673.152991773"
Jul 24 23:03:32.302486 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:03:32.302454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:04:03.495110 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:04:03.495061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:04:12.991223 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:04:12.991147    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:04:23.391921 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:04:23.391886    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:05:40.092667 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:05:40.092634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:05:41.251059 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:05:41.250960    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:06.303769 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:06.303733    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:06:16.938383 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:16.938343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:17.034498 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:17.034468    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:17.167745 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:17.167713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:23.113659 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:23.113628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:23.228103 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:23.228061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:23.331916 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:23.331882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:23.514894 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:23.514795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:29.410993 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:29.410956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:29.469285 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:29.469260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:06:45.995903 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:45.995860    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:06:55.791190 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:06:55.791153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:07:04.198984 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:04.198951    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:07:13.005163 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:13.005069    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:07:19.610789 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:19.610757    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:07:19.611253 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:19.610977    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:07:23.896755 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:23.896723    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:07:35.393141 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:35.393091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:07:47.090158 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:47.090112    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:07:56.688617 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:07:56.688582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:08:19.084725 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:08:19.084685    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:08:31.196212 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:08:31.196176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:08:43.837360 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:08:43.837284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:08:54.190225 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:08:54.190186    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:09:04.593005 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:09:04.592974    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:09:19.996500 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:09:19.996472    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:09:24.524422 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:09:24.524386    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:09:29.791782 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:09:29.791745    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:09:41.586583 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:09:41.586551    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:09:53.791524 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:09:53.791491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:10:24.581316 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:24.581281    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"]
Jul 24 23:10:24.582065 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:24.581485    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2" podUID="5b64bce8-c92b-49a7-8359-9809c1e01178" containerName="manager" containerID="cri-o://100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc" gracePeriod=10
Jul 24 23:10:28.145843 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.145817    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 23:10:28.235741 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.235672    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b64bce8-c92b-49a7-8359-9809c1e01178-extensions-socket-volume\") pod \"5b64bce8-c92b-49a7-8359-9809c1e01178\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") "
Jul 24 23:10:28.235741 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.235705    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b78d\" (UniqueName: \"kubernetes.io/projected/5b64bce8-c92b-49a7-8359-9809c1e01178-kube-api-access-5b78d\") pod \"5b64bce8-c92b-49a7-8359-9809c1e01178\" (UID: \"5b64bce8-c92b-49a7-8359-9809c1e01178\") "
Jul 24 23:10:28.236018 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.235998    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b64bce8-c92b-49a7-8359-9809c1e01178-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b64bce8-c92b-49a7-8359-9809c1e01178" (UID: "5b64bce8-c92b-49a7-8359-9809c1e01178"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:10:28.237643 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.237620    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b64bce8-c92b-49a7-8359-9809c1e01178-kube-api-access-5b78d" (OuterVolumeSpecName: "kube-api-access-5b78d") pod "5b64bce8-c92b-49a7-8359-9809c1e01178" (UID: "5b64bce8-c92b-49a7-8359-9809c1e01178"). InnerVolumeSpecName "kube-api-access-5b78d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:10:28.336965 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.336939    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b64bce8-c92b-49a7-8359-9809c1e01178-extensions-socket-volume\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:10:28.336965 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.336962    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b78d\" (UniqueName: \"kubernetes.io/projected/5b64bce8-c92b-49a7-8359-9809c1e01178-kube-api-access-5b78d\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:10:28.684629 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.684594    2564 generic.go:356] "Generic (PLEG): container finished" podID="5b64bce8-c92b-49a7-8359-9809c1e01178" containerID="100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc" exitCode=0
Jul 24 23:10:28.684790 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.684636    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"
Jul 24 23:10:28.684790 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.684669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2" event={"ID":"5b64bce8-c92b-49a7-8359-9809c1e01178","Type":"ContainerDied","Data":"100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc"}
Jul 24 23:10:28.684790 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.684713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2" event={"ID":"5b64bce8-c92b-49a7-8359-9809c1e01178","Type":"ContainerDied","Data":"b074da77ff76ae1939fc26020a381d4a119fcb2750806d3d215ed4ff854dbcaf"}
Jul 24 23:10:28.684790 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.684736    2564 scope.go:117] "RemoveContainer" containerID="100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc"
Jul 24 23:10:28.698755 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.698737    2564 scope.go:117] "RemoveContainer" containerID="100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc"
Jul 24 23:10:28.699004 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:10:28.698983    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc\": container with ID starting with 100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc not found: ID does not exist" containerID="100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc"
Jul 24 23:10:28.699054 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.699014    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc"} err="failed to get container status \"100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc\": rpc error: code = NotFound desc = could not find container \"100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc\": container with ID starting with 100d71f828c8848fd555e2dfd417c1e5cbc0500b1cd5f9f2e6d756cc1979cffc not found: ID does not exist"
Jul 24 23:10:28.708237 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.708216    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"]
Jul 24 23:10:28.714202 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:28.714183    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9xxw2"]
Jul 24 23:10:29.694981 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:10:29.694949    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b64bce8-c92b-49a7-8359-9809c1e01178" path="/var/lib/kubelet/pods/5b64bce8-c92b-49a7-8359-9809c1e01178/volumes"
Jul 24 23:11:25.714500 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.714469    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"]
Jul 24 23:11:25.715004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.714890    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b64bce8-c92b-49a7-8359-9809c1e01178" containerName="manager"
Jul 24 23:11:25.715004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.714901    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b64bce8-c92b-49a7-8359-9809c1e01178" containerName="manager"
Jul 24 23:11:25.715004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.714966    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b64bce8-c92b-49a7-8359-9809c1e01178" containerName="manager"
Jul 24 23:11:25.718828 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.718811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:25.721004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.720987    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-grhwh\""
Jul 24 23:11:25.728833 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.728804    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"]
Jul 24 23:11:25.821365 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.821333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc7f209e-c378-484d-83e5-b5f48097914b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sppcg\" (UID: \"dc7f209e-c378-484d-83e5-b5f48097914b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:25.821365 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.821370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f9pj\" (UniqueName: \"kubernetes.io/projected/dc7f209e-c378-484d-83e5-b5f48097914b-kube-api-access-2f9pj\") pod \"kuadrant-operator-controller-manager-86878c769f-sppcg\" (UID: \"dc7f209e-c378-484d-83e5-b5f48097914b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:25.922516 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.922480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc7f209e-c378-484d-83e5-b5f48097914b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sppcg\" (UID: \"dc7f209e-c378-484d-83e5-b5f48097914b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:25.922688 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.922527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f9pj\" (UniqueName: \"kubernetes.io/projected/dc7f209e-c378-484d-83e5-b5f48097914b-kube-api-access-2f9pj\") pod \"kuadrant-operator-controller-manager-86878c769f-sppcg\" (UID: \"dc7f209e-c378-484d-83e5-b5f48097914b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:25.922863 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.922843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc7f209e-c378-484d-83e5-b5f48097914b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sppcg\" (UID: \"dc7f209e-c378-484d-83e5-b5f48097914b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:25.934876 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:25.934850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f9pj\" (UniqueName: \"kubernetes.io/projected/dc7f209e-c378-484d-83e5-b5f48097914b-kube-api-access-2f9pj\") pod \"kuadrant-operator-controller-manager-86878c769f-sppcg\" (UID: \"dc7f209e-c378-484d-83e5-b5f48097914b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:26.030699 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:26.030609    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:26.192115 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:26.191861    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"]
Jul 24 23:11:26.194195 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:11:26.194163    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7f209e_c378_484d_83e5_b5f48097914b.slice/crio-1b8468bc548fd6370ba6fa17b1ce4064bb36855f1d7fb0b37e915c3f3a8509ac WatchSource:0}: Error finding container 1b8468bc548fd6370ba6fa17b1ce4064bb36855f1d7fb0b37e915c3f3a8509ac: Status 404 returned error can't find the container with id 1b8468bc548fd6370ba6fa17b1ce4064bb36855f1d7fb0b37e915c3f3a8509ac
Jul 24 23:11:27.002263 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:27.002230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg" event={"ID":"dc7f209e-c378-484d-83e5-b5f48097914b","Type":"ContainerStarted","Data":"6e89924ab66abbbddff84f8bed38c67752d7e2445e2f5d35e853243042666741"}
Jul 24 23:11:27.002639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:27.002280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg" event={"ID":"dc7f209e-c378-484d-83e5-b5f48097914b","Type":"ContainerStarted","Data":"1b8468bc548fd6370ba6fa17b1ce4064bb36855f1d7fb0b37e915c3f3a8509ac"}
Jul 24 23:11:27.002639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:27.002430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:27.022973 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:27.022928    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg" podStartSLOduration=2.022914728 podStartE2EDuration="2.022914728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:11:27.021268708 +0000 UTC m=+1147.874595248" watchObservedRunningTime="2026-07-24 23:11:27.022914728 +0000 UTC m=+1147.876241268"
Jul 24 23:11:38.009601 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:38.009571    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sppcg"
Jul 24 23:11:47.893720 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:47.893683    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:11:59.885944 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:11:59.885900    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:12:08.691214 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:08.691178    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:12:12.876818 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:12.876786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:12:19.653305 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:19.653276    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:12:19.656309 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:19.656290    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:12:22.389239 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:22.389203    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:12:31.588218 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:31.588181    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:12:42.488162 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:42.488116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:12:51.389494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:12:51.389458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:13:02.193443 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:13:02.193403    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:13:13.695800 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:13:13.695717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:13:22.491076 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:13:22.491043    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:13:33.890762 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:13:33.890729    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:13:56.689379 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:13:56.689344    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:14:05.887906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:14:05.887875    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:14:07.340942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:14:07.340836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:14:17.490855 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:14:17.490818    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:14:27.911007 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:14:27.910977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:14:31.495060 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:14:31.495024    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:15:12.787968 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:15:12.787936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:15:13.513877 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:15:13.513845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:15:16.492370 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:15:16.492337    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:15:30.397322 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:15:30.397277    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:15:36.683511 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:15:36.683476    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:15:51.792816 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:15:51.792779    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:16:01.596229 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:16:01.596192    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:16:11.086613 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:16:11.086582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:16:21.388096 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:16:21.388063    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:16:30.303189 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:16:30.303146    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:16:39.387567 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:16:39.387532    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:16:51.890053 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:16:51.890009    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:17:01.656323 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:01.656286    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:17:13.592449 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:13.592411    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:17:19.694292 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:19.694258    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:17:19.698617 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:19.698592    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:17:25.695133 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:25.695105    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:17:31.052225 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:31.052192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:17:31.139681 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:31.139639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:17:36.793603 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:36.793568    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:17:45.800247 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:45.800214    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:17:58.698728 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:17:58.698689    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:01.447420 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:01.447381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:18:08.216727 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:08.216694    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:17.984236 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:17.984200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:26.694596 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:26.694560    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:29.933669 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:29.933629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:18:35.992307 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:35.992272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:43.106147 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:43.106088    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:18:47.097426 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:47.097392    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:53.449633 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:53.449600    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:18:56.488332 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:56.488299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:18:56.539592 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:56.539548    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:18:56.639396 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:56.639355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:18:57.060149 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:18:57.060096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:02.758380 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:02.758346    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:02.859757 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:02.859724    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:02.994061 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:02.994031    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:03.312900 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:03.312848    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:03.772471 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:03.772444    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:05.194184 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:05.194146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:05.295350 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:05.295319    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:05.375264 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:05.375231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:05.873790 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:05.873763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:05.973019 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:05.972981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:06.100117 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:06.100085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:13.089549 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:13.089466    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:19:41.185042 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:41.185007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:42.241711 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:42.241675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:44.289606 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:44.289573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:19:45.850090 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:45.850022    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:46.007034 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:46.007006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:46.227678 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:46.227575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:46.483759 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:46.483644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:46.582144 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:46.582098    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:46.696274 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:46.696233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:47.003982 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:47.003952    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:47.262957 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:47.262859    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:49.516574 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:49.516539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:49.817646 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:49.817543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:50.114194 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:50.114167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:50.295858 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:50.295826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:50.392478 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:50.392398    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:19:54.687442 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:19:54.687403    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:20:24.944869 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:24.944826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:25.044225 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:25.044193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:25.093808 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:25.093776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:28.582335 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:28.582298    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:20:29.609639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:29.609605    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:29.769569 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:29.769533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:30.011468 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:30.011388    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:30.240056 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:30.240015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:30.582339 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:30.582289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:30.791479 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:30.791440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:32.618295 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:32.618266    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:32.720923 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:32.720891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:32.838441 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:32.838402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:33.010175 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:33.010041    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:33.389616 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:33.389566    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:33.458137 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:33.458066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:20:39.194421 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:20:39.194385    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:21:09.350355 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:09.350319    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:09.416193 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:09.416161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:09.469938 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:09.469901    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:12.092570 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:12.092528    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:21:13.567808 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:13.567727    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:14.790614 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:14.790578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:15.015849 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:15.015817    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:15.263025 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:15.262988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:15.550284 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:15.550207    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:15.738650 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:15.738607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:17.822141 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:17.822038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:17.905695 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:17.905649    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:17.977228 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:17.977175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:18.074838 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:18.074739    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:18.492988 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:18.492953    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:18.587437 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:18.587407    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:23.393066 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:23.393025    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:21:41.945573 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:41.945541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:42.000993 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:42.000962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:53.141640 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:53.141596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:53.204757 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:53.204720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:53.262049 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:53.262017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:56.196330 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:56.196297    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:21:57.788273 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:57.788237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:57.945449 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:57.945420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:58.193656 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:58.193625    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:58.689803 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:58.689767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:58.780308 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:58.780263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:21:58.926561 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:21:58.926517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:00.331021 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:00.330988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:06.322198 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:06.322157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:06.427443 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:06.427411    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:06.523983 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:06.523945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:06.627168 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:06.627116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:07.088793 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:07.088761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:11.103186 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:11.103146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:15.715737 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:15.715709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:19.731656 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:19.731628    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:22:19.736373 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:19.736350    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:22:33.037694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:33.037659    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:39.420144 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:39.420090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:39.520906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:39.520874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:39.619332 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:39.619296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:22:55.507048 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:22:55.507012    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:01.109431 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:01.109395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:01.217706 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:01.217668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:01.309963 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:01.309927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:11.865797 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:11.865763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:12.662597 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:12.662568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:19.034389 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:19.034355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:19.135975 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:19.135945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:19.236786 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:19.236748    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:19.332741 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:19.332698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:20.157902 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:20.157813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:20.252942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:20.252896    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:21.125613 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:21.125561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:21.184173 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:21.184110    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:21.908911 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:21.908879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:22.992950 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:22.992914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:23:28.551391 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:28.551350    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:28.654138 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:28.654100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:29.289631 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:29.289595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:32.893573 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:32.893538    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:23:35.427255 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:35.427188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:35.492257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:35.492191    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:35.677373 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:35.677286    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:35.995184 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:35.995100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:36.221404 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:36.221355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:39.057926 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:39.057866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:39.153697 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:39.153668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:39.224494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:39.224460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:39.545612 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:39.545578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:39.879764 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:39.879720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:39.979824 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:39.979788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:43.405030 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:43.404947    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:23:43.556695 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:43.556660    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:44.211723 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:44.211685    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:44.315718 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:44.315681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:44.661839 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:44.661803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:47.814753 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:47.814720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:50.612862 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:50.612825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:23:50.712862 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:23:50.712826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:01.130466 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:01.130428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:01.230461 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:01.230434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:01.339396 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:01.339359    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:04.194605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:04.194570    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:24:06.075006 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:06.074963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:06.455619 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:06.455587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:12.630541 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:12.630509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:12.730614 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:12.730582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:12.906197 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:12.906113    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:13.140633 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:13.140587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:13.267848 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:13.267809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:13.367380 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:13.367345    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:13.483541 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:13.483514    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:19.303317 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:19.303267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:19.404005 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:19.403976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:19.498718 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:19.498686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:19.655229 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:19.655195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:25.509291 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:25.509257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:25.576284 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:25.576247    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:32.377398 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:32.377364    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:32.833852 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:32.833814    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:32.933990 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:32.933962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:35.346294 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:35.346241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:35.433785 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:35.433750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:35.627927 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:35.627835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:36.130023 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:36.129987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:36.243584 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:36.243553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:36.375998 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:36.375961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:39.829899 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:39.829872    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:39.994414 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:39.994383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:24:58.187709 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.187675    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"]
Jul 24 23:24:58.191939 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.191913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.195490 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.195468    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9d44355\"/\"kube-root-ca.crt\""
Jul 24 23:24:58.195701 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.195681    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9d44355\"/\"default-dockercfg-pw9jl\""
Jul 24 23:24:58.195782 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.195713    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9d44355\"/\"openshift-service-ca.crt\""
Jul 24 23:24:58.195959 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.195941    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9d44355\"/\"e2e-lifecycle-model-e9d44355-kserve-self-signed-certs\""
Jul 24 23:24:58.199434 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.199409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"]
Jul 24 23:24:58.225434 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.225408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:24:58.329776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.329749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62a89d5-df1d-4222-810a-da170345dd07-tls-certs\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.329936 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.329781    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-dshm\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.329936 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.329815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6r6l\" (UniqueName: \"kubernetes.io/projected/b62a89d5-df1d-4222-810a-da170345dd07-kube-api-access-l6r6l\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.329936 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.329864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-home\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.330066 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.329957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-tmp-dir\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.330066 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.329996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-model-cache\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431331 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-model-cache\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431478 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62a89d5-df1d-4222-810a-da170345dd07-tls-certs\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431478 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431379    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-dshm\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431478 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6r6l\" (UniqueName: \"kubernetes.io/projected/b62a89d5-df1d-4222-810a-da170345dd07-kube-api-access-l6r6l\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431478 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-home\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431701 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-tmp-dir\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431761 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431714    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-model-cache\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431892 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-home\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.431945 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.431901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-tmp-dir\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.433951 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.433925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-dshm\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.434055 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.433997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62a89d5-df1d-4222-810a-da170345dd07-tls-certs\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.439888 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.439832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6r6l\" (UniqueName: \"kubernetes.io/projected/b62a89d5-df1d-4222-810a-da170345dd07-kube-api-access-l6r6l\") pod \"e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") " pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.504197 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.504176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:24:58.639447 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.639415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"]
Jul 24 23:24:58.639967 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:24:58.639946    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb62a89d5_df1d_4222_810a_da170345dd07.slice/crio-3b16dd12e313817fc2769b9a5e1d767667da16038e3a4a0bed76c14d8645b274 WatchSource:0}: Error finding container 3b16dd12e313817fc2769b9a5e1d767667da16038e3a4a0bed76c14d8645b274: Status 404 returned error can't find the container with id 3b16dd12e313817fc2769b9a5e1d767667da16038e3a4a0bed76c14d8645b274
Jul 24 23:24:58.741254 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.741219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h" event={"ID":"b62a89d5-df1d-4222-810a-da170345dd07","Type":"ContainerStarted","Data":"9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4"}
Jul 24 23:24:58.741254 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.741259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h" event={"ID":"b62a89d5-df1d-4222-810a-da170345dd07","Type":"ContainerStarted","Data":"3b16dd12e313817fc2769b9a5e1d767667da16038e3a4a0bed76c14d8645b274"}
Jul 24 23:24:58.758679 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:24:58.758636    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h" podStartSLOduration=0.758622126 podStartE2EDuration="758.622126ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:24:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:24:58.756795152 +0000 UTC m=+1959.610121692" watchObservedRunningTime="2026-07-24 23:24:58.758622126 +0000 UTC m=+1959.611948661"
Jul 24 23:25:01.292890 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:01.292854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:25:06.392022 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:06.391992    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:25:08.504555 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:08.504522    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:25:08.504919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:08.504567    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:25:08.516950 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:08.516929    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:25:08.808377 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:08.808305    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:25:14.890250 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:14.890215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:25:33.899181 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:33.899146    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:25:35.039056 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:35.039014    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:25:35.137997 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:35.137946    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:25:35.240328 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:35.240296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:25:39.526091 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:39.526057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:25:42.398541 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:42.398508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:25:42.782340 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:42.782244    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:25:47.758318 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:47.758286    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"]
Jul 24 23:25:47.802165 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:25:47.802110    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:02.777434 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:02.777392    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h" podUID="b62a89d5-df1d-4222-810a-da170345dd07" containerName="main" containerID="cri-o://9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4" gracePeriod=45
Jul 24 23:26:03.029526 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.029467    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:26:03.107663 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.107628    2564 generic.go:356] "Generic (PLEG): container finished" podID="b62a89d5-df1d-4222-810a-da170345dd07" containerID="9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4" exitCode=0
Jul 24 23:26:03.107834 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.107668    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"
Jul 24 23:26:03.107834 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.107667    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h" event={"ID":"b62a89d5-df1d-4222-810a-da170345dd07","Type":"ContainerDied","Data":"9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4"}
Jul 24 23:26:03.107834 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.107716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h" event={"ID":"b62a89d5-df1d-4222-810a-da170345dd07","Type":"ContainerDied","Data":"3b16dd12e313817fc2769b9a5e1d767667da16038e3a4a0bed76c14d8645b274"}
Jul 24 23:26:03.107834 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.107734    2564 scope.go:117] "RemoveContainer" containerID="9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4"
Jul 24 23:26:03.116610 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.116591    2564 scope.go:117] "RemoveContainer" containerID="9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4"
Jul 24 23:26:03.116893 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:26:03.116875    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4\": container with ID starting with 9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4 not found: ID does not exist" containerID="9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4"
Jul 24 23:26:03.116982 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.116903    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4"} err="failed to get container status \"9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4\": rpc error: code = NotFound desc = could not find container \"9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4\": container with ID starting with 9233b02f571e5d286c5997b7565f4f025f173db3245ab5a5ec40fa3e64d9f1d4 not found: ID does not exist"
Jul 24 23:26:03.178116 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178088    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62a89d5-df1d-4222-810a-da170345dd07-tls-certs\") pod \"b62a89d5-df1d-4222-810a-da170345dd07\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") "
Jul 24 23:26:03.178291 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178150    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-dshm\") pod \"b62a89d5-df1d-4222-810a-da170345dd07\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") "
Jul 24 23:26:03.178291 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178266    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-tmp-dir\") pod \"b62a89d5-df1d-4222-810a-da170345dd07\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") "
Jul 24 23:26:03.178407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178334    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-model-cache\") pod \"b62a89d5-df1d-4222-810a-da170345dd07\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") "
Jul 24 23:26:03.178407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178366    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-home\") pod \"b62a89d5-df1d-4222-810a-da170345dd07\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") "
Jul 24 23:26:03.178512 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178449    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6r6l\" (UniqueName: \"kubernetes.io/projected/b62a89d5-df1d-4222-810a-da170345dd07-kube-api-access-l6r6l\") pod \"b62a89d5-df1d-4222-810a-da170345dd07\" (UID: \"b62a89d5-df1d-4222-810a-da170345dd07\") "
Jul 24 23:26:03.178562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178535    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b62a89d5-df1d-4222-810a-da170345dd07" (UID: "b62a89d5-df1d-4222-810a-da170345dd07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:26:03.178617 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-model-cache" (OuterVolumeSpecName: "model-cache") pod "b62a89d5-df1d-4222-810a-da170345dd07" (UID: "b62a89d5-df1d-4222-810a-da170345dd07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:26:03.178690 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178662    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-home" (OuterVolumeSpecName: "home") pod "b62a89d5-df1d-4222-810a-da170345dd07" (UID: "b62a89d5-df1d-4222-810a-da170345dd07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:26:03.178841 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178821    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:26:03.178911 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178856    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:26:03.178911 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.178872    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:26:03.180309 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.180287    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b62a89d5-df1d-4222-810a-da170345dd07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b62a89d5-df1d-4222-810a-da170345dd07" (UID: "b62a89d5-df1d-4222-810a-da170345dd07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:26:03.180402 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.180354    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-dshm" (OuterVolumeSpecName: "dshm") pod "b62a89d5-df1d-4222-810a-da170345dd07" (UID: "b62a89d5-df1d-4222-810a-da170345dd07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:26:03.180402 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.180395    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62a89d5-df1d-4222-810a-da170345dd07-kube-api-access-l6r6l" (OuterVolumeSpecName: "kube-api-access-l6r6l") pod "b62a89d5-df1d-4222-810a-da170345dd07" (UID: "b62a89d5-df1d-4222-810a-da170345dd07"). InnerVolumeSpecName "kube-api-access-l6r6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:26:03.280098 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.280021    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6r6l\" (UniqueName: \"kubernetes.io/projected/b62a89d5-df1d-4222-810a-da170345dd07-kube-api-access-l6r6l\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:26:03.280098 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.280047    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62a89d5-df1d-4222-810a-da170345dd07-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:26:03.280098 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.280060    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62a89d5-df1d-4222-810a-da170345dd07-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:26:03.430940 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.430913    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"]
Jul 24 23:26:03.434586 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.434563    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e9d44355/e2e-lifecycle-model-e9d44355-kserve-765589f9cf-q875h"]
Jul 24 23:26:03.695099 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:03.695069    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b62a89d5-df1d-4222-810a-da170345dd07" path="/var/lib/kubelet/pods/b62a89d5-df1d-4222-810a-da170345dd07/volumes"
Jul 24 23:26:23.894856 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:23.894794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:25.162785 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:25.162742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:31.209605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:31.209575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:31.309999 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:31.309967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:31.532000 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:31.531919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:31.766466 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:31.766435    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:32.862870 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:32.862823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:32.951567 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:32.951516    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:33.052677 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:33.052638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:33.895970 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:33.895944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:34.002047 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:34.002011    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:35.318446 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:35.318414    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:26:35.591354 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:35.591325    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:41.849647 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:41.849564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:41.952213 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:41.952181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:42.585361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:42.585327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:45.209329 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:45.209300    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:26:49.110849 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:49.110819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:49.215661 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:49.215624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:49.310899 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:49.310862    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:26:59.847674 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:26:59.847638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:00.246708 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:00.246618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:19.767506 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:19.767480    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:27:19.775334 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:19.775311    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:27:21.173261 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:21.173210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:27.472400 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:27.472347    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:27.573242 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:27.573118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:27.809474 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:27.809366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:28.117889 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:28.117832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:28.366964 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:28.366926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:31.075760 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:31.075722    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:31.174996 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:31.174960    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:31.346792 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:31.346750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:31.633805 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:31.633722    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:31.885415 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:31.885341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:31.985992 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:31.985950    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:35.703254 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:35.703225    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:35.950397 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:35.950368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:36.254752 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:36.254722    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:36.593923 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:36.593893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:36.806065 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:36.806034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:27:39.125588 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:39.125542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:39.225053 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:39.225019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:39.309885 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:39.309853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:39.647708 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:39.647682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:39.914088 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:39.914010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:40.010985 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:40.010954    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:43.636638 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:43.636607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:45.694452 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:45.694422    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:27:58.991768 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.991733    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"]
Jul 24 23:27:58.994000 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.992410    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b62a89d5-df1d-4222-810a-da170345dd07" containerName="main"
Jul 24 23:27:58.994000 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.992429    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b62a89d5-df1d-4222-810a-da170345dd07" containerName="main"
Jul 24 23:27:58.994000 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.992538    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b62a89d5-df1d-4222-810a-da170345dd07" containerName="main"
Jul 24 23:27:58.995931 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.995914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:58.999768 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.999747    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-eb73b6\"/\"default-dockercfg-nbkfn\""
Jul 24 23:27:58.999974 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:58.999959    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-eb73b6\"/\"test-model-eb73b6-kserve-self-signed-certs\""
Jul 24 23:27:59.000179 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.000164    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-eb73b6\"/\"openshift-service-ca.crt\""
Jul 24 23:27:59.000361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.000343    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-eb73b6\"/\"kube-root-ca.crt\""
Jul 24 23:27:59.003639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.003581    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"]
Jul 24 23:27:59.023912 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.023888    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:27:59.081350 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.081321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-dshm\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.081477 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.081370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp8hd\" (UniqueName: \"kubernetes.io/projected/c30f3b73-6432-499d-9517-98b5bb187982-kube-api-access-bp8hd\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.081477 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.081424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-home\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.081477 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.081463    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30f3b73-6432-499d-9517-98b5bb187982-tls-certs\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.081579 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.081504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-model-cache\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.081619 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.081585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-tmp-dir\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.182881 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.182853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-model-cache\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183036 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.182912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-tmp-dir\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183036 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.182939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-dshm\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183036 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.182960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp8hd\" (UniqueName: \"kubernetes.io/projected/c30f3b73-6432-499d-9517-98b5bb187982-kube-api-access-bp8hd\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183036 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.182980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-home\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183036 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.182998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30f3b73-6432-499d-9517-98b5bb187982-tls-certs\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183328 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.183312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-model-cache\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183422 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.183394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-tmp-dir\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.183522 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.183447    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-home\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.185223 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.185201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-dshm\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.185487 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.185471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30f3b73-6432-499d-9517-98b5bb187982-tls-certs\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.191517 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.191495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp8hd\" (UniqueName: \"kubernetes.io/projected/c30f3b73-6432-499d-9517-98b5bb187982-kube-api-access-bp8hd\") pod \"test-model-eb73b6-kserve-646578c79-r92rp\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") " pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.307241 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.307173    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:27:59.643401 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.643381    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"]
Jul 24 23:27:59.645140 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:27:59.645096    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc30f3b73_6432_499d_9517_98b5bb187982.slice/crio-185a1aca25427a20034163f7fa417cf4623a910fd8a8cb76e2ed4c63edcd3d84 WatchSource:0}: Error finding container 185a1aca25427a20034163f7fa417cf4623a910fd8a8cb76e2ed4c63edcd3d84: Status 404 returned error can't find the container with id 185a1aca25427a20034163f7fa417cf4623a910fd8a8cb76e2ed4c63edcd3d84
Jul 24 23:27:59.778061 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.778027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp" event={"ID":"c30f3b73-6432-499d-9517-98b5bb187982","Type":"ContainerStarted","Data":"0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48"}
Jul 24 23:27:59.778238 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.778073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp" event={"ID":"c30f3b73-6432-499d-9517-98b5bb187982","Type":"ContainerStarted","Data":"185a1aca25427a20034163f7fa417cf4623a910fd8a8cb76e2ed4c63edcd3d84"}
Jul 24 23:27:59.797540 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:27:59.797496    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp" podStartSLOduration=1.797480789 podStartE2EDuration="1.797480789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:27:59.7950014 +0000 UTC m=+2140.648327957" watchObservedRunningTime="2026-07-24 23:27:59.797480789 +0000 UTC m=+2140.650807307"
Jul 24 23:28:03.193450 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:03.193415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:28:09.308210 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:09.308177    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:28:09.308656 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:09.308225    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:28:09.320419 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:09.320395    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:28:09.844774 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:09.844737    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:28:30.397410 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.397377    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"]
Jul 24 23:28:30.401056 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.401037    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.404766 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.404745    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0f3852\"/\"openshift-service-ca.crt\""
Jul 24 23:28:30.404879 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.404785    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0f3852\"/\"default-dockercfg-qggzg\""
Jul 24 23:28:30.404944 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.404875    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0f3852\"/\"test-model-0f3852-kserve-self-signed-certs\""
Jul 24 23:28:30.404944 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.404890    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0f3852\"/\"kube-root-ca.crt\""
Jul 24 23:28:30.410897 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.410872    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"]
Jul 24 23:28:30.424162 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.424141    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:28:30.448476 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.448453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-model-cache\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.448577 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.448530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-home\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.448631 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.448583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-tmp-dir\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.448684 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.448646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-dshm\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.448684 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.448671    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1032f1-7b09-452a-98ed-05f59629d0c8-tls-certs\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.448765 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.448697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fssx\" (UniqueName: \"kubernetes.io/projected/9d1032f1-7b09-452a-98ed-05f59629d0c8-kube-api-access-9fssx\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.549904 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.549877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-tmp-dir\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550093 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.549928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-dshm\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550093 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.549955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1032f1-7b09-452a-98ed-05f59629d0c8-tls-certs\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550093 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.549983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fssx\" (UniqueName: \"kubernetes.io/projected/9d1032f1-7b09-452a-98ed-05f59629d0c8-kube-api-access-9fssx\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550093 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.550036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-model-cache\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550349 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.550112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-home\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550409 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.550346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-tmp-dir\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550490 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.550461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-model-cache\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.550541 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.550486    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-home\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.552218 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.552197    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-dshm\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.552392 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.552376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1032f1-7b09-452a-98ed-05f59629d0c8-tls-certs\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.557990 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.557968    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fssx\" (UniqueName: \"kubernetes.io/projected/9d1032f1-7b09-452a-98ed-05f59629d0c8-kube-api-access-9fssx\") pod \"test-model-0f3852-kserve-65f479595c-9wmhg\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") " pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.711561 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.711480    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:30.856188 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.856158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"]
Jul 24 23:28:30.857625 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:28:30.857601    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d1032f1_7b09_452a_98ed_05f59629d0c8.slice/crio-4e5563f7066a054353c09d13ddbf5d33ba83e3b91d377957c38114b9e7670b34 WatchSource:0}: Error finding container 4e5563f7066a054353c09d13ddbf5d33ba83e3b91d377957c38114b9e7670b34: Status 404 returned error can't find the container with id 4e5563f7066a054353c09d13ddbf5d33ba83e3b91d377957c38114b9e7670b34
Jul 24 23:28:30.958097 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.958062    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg" event={"ID":"9d1032f1-7b09-452a-98ed-05f59629d0c8","Type":"ContainerStarted","Data":"efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd"}
Jul 24 23:28:30.958270 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.958111    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg" event={"ID":"9d1032f1-7b09-452a-98ed-05f59629d0c8","Type":"ContainerStarted","Data":"4e5563f7066a054353c09d13ddbf5d33ba83e3b91d377957c38114b9e7670b34"}
Jul 24 23:28:30.978436 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:30.978351    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg" podStartSLOduration=0.97833632 podStartE2EDuration="978.33632ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:28:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:28:30.975100155 +0000 UTC m=+2171.828426693" watchObservedRunningTime="2026-07-24 23:28:30.97833632 +0000 UTC m=+2171.831662866"
Jul 24 23:28:35.595481 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:35.595440    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:28:40.712233 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:40.712195    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:40.712233 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:40.712243    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:40.724838 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:40.724816    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:28:41.025436 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:28:41.025366    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:29:23.917260 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:23.917227    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"]
Jul 24 23:29:23.942573 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:23.942520    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:24.826372 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:24.826339    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"]
Jul 24 23:29:24.847449 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:24.847418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:28.390368 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:28.390333    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:29:35.403312 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:35.403282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:35.463648 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:35.463613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:38.936410 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:38.936371    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp" podUID="c30f3b73-6432-499d-9517-98b5bb187982" containerName="main" containerID="cri-o://0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48" gracePeriod=45
Jul 24 23:29:39.184920 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.184900    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:29:39.239943 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.239876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-tmp-dir\") pod \"c30f3b73-6432-499d-9517-98b5bb187982\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") "
Jul 24 23:29:39.239943 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.239924    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30f3b73-6432-499d-9517-98b5bb187982-tls-certs\") pod \"c30f3b73-6432-499d-9517-98b5bb187982\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") "
Jul 24 23:29:39.240152 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.239968    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp8hd\" (UniqueName: \"kubernetes.io/projected/c30f3b73-6432-499d-9517-98b5bb187982-kube-api-access-bp8hd\") pod \"c30f3b73-6432-499d-9517-98b5bb187982\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") "
Jul 24 23:29:39.240152 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240002    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-model-cache\") pod \"c30f3b73-6432-499d-9517-98b5bb187982\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") "
Jul 24 23:29:39.240152 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240051    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-dshm\") pod \"c30f3b73-6432-499d-9517-98b5bb187982\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") "
Jul 24 23:29:39.240152 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-home\") pod \"c30f3b73-6432-499d-9517-98b5bb187982\" (UID: \"c30f3b73-6432-499d-9517-98b5bb187982\") "
Jul 24 23:29:39.240353 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240155    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c30f3b73-6432-499d-9517-98b5bb187982" (UID: "c30f3b73-6432-499d-9517-98b5bb187982"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:39.240407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240387    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:39.240550 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240529    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-home" (OuterVolumeSpecName: "home") pod "c30f3b73-6432-499d-9517-98b5bb187982" (UID: "c30f3b73-6432-499d-9517-98b5bb187982"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:39.240655 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.240633    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-model-cache" (OuterVolumeSpecName: "model-cache") pod "c30f3b73-6432-499d-9517-98b5bb187982" (UID: "c30f3b73-6432-499d-9517-98b5bb187982"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:39.242412 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.242383    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-dshm" (OuterVolumeSpecName: "dshm") pod "c30f3b73-6432-499d-9517-98b5bb187982" (UID: "c30f3b73-6432-499d-9517-98b5bb187982"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:39.242536 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.242482    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30f3b73-6432-499d-9517-98b5bb187982-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c30f3b73-6432-499d-9517-98b5bb187982" (UID: "c30f3b73-6432-499d-9517-98b5bb187982"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:29:39.242587 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.242564    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30f3b73-6432-499d-9517-98b5bb187982-kube-api-access-bp8hd" (OuterVolumeSpecName: "kube-api-access-bp8hd") pod "c30f3b73-6432-499d-9517-98b5bb187982" (UID: "c30f3b73-6432-499d-9517-98b5bb187982"). InnerVolumeSpecName "kube-api-access-bp8hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:39.341381 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.341341    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp8hd\" (UniqueName: \"kubernetes.io/projected/c30f3b73-6432-499d-9517-98b5bb187982-kube-api-access-bp8hd\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:39.341381 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.341377    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:39.341381 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.341388    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:39.341381 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.341395    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30f3b73-6432-499d-9517-98b5bb187982-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:39.341607 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.341403    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30f3b73-6432-499d-9517-98b5bb187982-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:39.356197 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.356169    2564 generic.go:356] "Generic (PLEG): container finished" podID="c30f3b73-6432-499d-9517-98b5bb187982" containerID="0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48" exitCode=0
Jul 24 23:29:39.356318 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.356223    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"
Jul 24 23:29:39.356318 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.356242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp" event={"ID":"c30f3b73-6432-499d-9517-98b5bb187982","Type":"ContainerDied","Data":"0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48"}
Jul 24 23:29:39.356318 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.356302    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp" event={"ID":"c30f3b73-6432-499d-9517-98b5bb187982","Type":"ContainerDied","Data":"185a1aca25427a20034163f7fa417cf4623a910fd8a8cb76e2ed4c63edcd3d84"}
Jul 24 23:29:39.356437 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.356325    2564 scope.go:117] "RemoveContainer" containerID="0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48"
Jul 24 23:29:39.366069 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.366052    2564 scope.go:117] "RemoveContainer" containerID="0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48"
Jul 24 23:29:39.366363 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:29:39.366347    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48\": container with ID starting with 0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48 not found: ID does not exist" containerID="0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48"
Jul 24 23:29:39.366430 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.366371    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48"} err="failed to get container status \"0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48\": rpc error: code = NotFound desc = could not find container \"0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48\": container with ID starting with 0a80a22660b99f50cda77fea8b9424f60581b0b3f678100c5f421838a7be3e48 not found: ID does not exist"
Jul 24 23:29:39.376715 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.376688    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"]
Jul 24 23:29:39.379071 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.379051    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-eb73b6/test-model-eb73b6-kserve-646578c79-r92rp"]
Jul 24 23:29:39.694933 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.694904    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30f3b73-6432-499d-9517-98b5bb187982" path="/var/lib/kubelet/pods/c30f3b73-6432-499d-9517-98b5bb187982/volumes"
Jul 24 23:29:39.846486 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:39.846441    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg" podUID="9d1032f1-7b09-452a-98ed-05f59629d0c8" containerName="main" containerID="cri-o://efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd" gracePeriod=45
Jul 24 23:29:40.042013 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.041979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:40.093984 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.093963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:29:40.149275 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149247    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-tmp-dir\") pod \"9d1032f1-7b09-452a-98ed-05f59629d0c8\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") "
Jul 24 23:29:40.149275 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149275    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1032f1-7b09-452a-98ed-05f59629d0c8-tls-certs\") pod \"9d1032f1-7b09-452a-98ed-05f59629d0c8\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") "
Jul 24 23:29:40.149494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149304    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-model-cache\") pod \"9d1032f1-7b09-452a-98ed-05f59629d0c8\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") "
Jul 24 23:29:40.149494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fssx\" (UniqueName: \"kubernetes.io/projected/9d1032f1-7b09-452a-98ed-05f59629d0c8-kube-api-access-9fssx\") pod \"9d1032f1-7b09-452a-98ed-05f59629d0c8\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") "
Jul 24 23:29:40.149494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149365    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-home\") pod \"9d1032f1-7b09-452a-98ed-05f59629d0c8\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") "
Jul 24 23:29:40.149494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149462    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-dshm\") pod \"9d1032f1-7b09-452a-98ed-05f59629d0c8\" (UID: \"9d1032f1-7b09-452a-98ed-05f59629d0c8\") "
Jul 24 23:29:40.149692 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149549    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d1032f1-7b09-452a-98ed-05f59629d0c8" (UID: "9d1032f1-7b09-452a-98ed-05f59629d0c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:40.149692 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d1032f1-7b09-452a-98ed-05f59629d0c8" (UID: "9d1032f1-7b09-452a-98ed-05f59629d0c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:40.149772 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149687    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-home" (OuterVolumeSpecName: "home") pod "9d1032f1-7b09-452a-98ed-05f59629d0c8" (UID: "9d1032f1-7b09-452a-98ed-05f59629d0c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:40.149840 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149798    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:40.149840 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149809    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:40.149840 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.149818    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:40.151429 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.151407    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d1032f1-7b09-452a-98ed-05f59629d0c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d1032f1-7b09-452a-98ed-05f59629d0c8" (UID: "9d1032f1-7b09-452a-98ed-05f59629d0c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:29:40.151494 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.151463    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-dshm" (OuterVolumeSpecName: "dshm") pod "9d1032f1-7b09-452a-98ed-05f59629d0c8" (UID: "9d1032f1-7b09-452a-98ed-05f59629d0c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:29:40.151539 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.151528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1032f1-7b09-452a-98ed-05f59629d0c8-kube-api-access-9fssx" (OuterVolumeSpecName: "kube-api-access-9fssx") pod "9d1032f1-7b09-452a-98ed-05f59629d0c8" (UID: "9d1032f1-7b09-452a-98ed-05f59629d0c8"). InnerVolumeSpecName "kube-api-access-9fssx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:40.250884 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.250816    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1032f1-7b09-452a-98ed-05f59629d0c8-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:40.250884 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.250837    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fssx\" (UniqueName: \"kubernetes.io/projected/9d1032f1-7b09-452a-98ed-05f59629d0c8-kube-api-access-9fssx\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:40.250884 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.250846    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1032f1-7b09-452a-98ed-05f59629d0c8-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:40.363735 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.363700    2564 generic.go:356] "Generic (PLEG): container finished" podID="9d1032f1-7b09-452a-98ed-05f59629d0c8" containerID="efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd" exitCode=0
Jul 24 23:29:40.363892 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.363761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg" event={"ID":"9d1032f1-7b09-452a-98ed-05f59629d0c8","Type":"ContainerDied","Data":"efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd"}
Jul 24 23:29:40.363892 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.363770    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"
Jul 24 23:29:40.363892 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.363792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg" event={"ID":"9d1032f1-7b09-452a-98ed-05f59629d0c8","Type":"ContainerDied","Data":"4e5563f7066a054353c09d13ddbf5d33ba83e3b91d377957c38114b9e7670b34"}
Jul 24 23:29:40.363892 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.363809    2564 scope.go:117] "RemoveContainer" containerID="efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd"
Jul 24 23:29:40.373205 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.373189    2564 scope.go:117] "RemoveContainer" containerID="efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd"
Jul 24 23:29:40.373449 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:29:40.373429    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd\": container with ID starting with efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd not found: ID does not exist" containerID="efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd"
Jul 24 23:29:40.373493 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.373458    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd"} err="failed to get container status \"efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd\": rpc error: code = NotFound desc = could not find container \"efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd\": container with ID starting with efe0bf8600042e6d5410dfae8334977e0a8f7528dacf9db62901d518c3ac2ccd not found: ID does not exist"
Jul 24 23:29:40.384829 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.384807    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"]
Jul 24 23:29:40.387373 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.387355    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:29:40.390293 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.390275    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0f3852/test-model-0f3852-kserve-65f479595c-9wmhg"]
Jul 24 23:29:40.422623 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:40.422603    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:41.695142 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:41.695050    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d1032f1-7b09-452a-98ed-05f59629d0c8" path="/var/lib/kubelet/pods/9d1032f1-7b09-452a-98ed-05f59629d0c8/volumes"
Jul 24 23:29:46.248756 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.248729    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.348985 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.348955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.527822 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.527738    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.743829 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.743798    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.868583 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.868553    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"]
Jul 24 23:29:46.869216 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.869200    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30f3b73-6432-499d-9517-98b5bb187982" containerName="main"
Jul 24 23:29:46.869268 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.869218    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30f3b73-6432-499d-9517-98b5bb187982" containerName="main"
Jul 24 23:29:46.869268 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.869246    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1032f1-7b09-452a-98ed-05f59629d0c8" containerName="main"
Jul 24 23:29:46.869268 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.869254    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1032f1-7b09-452a-98ed-05f59629d0c8" containerName="main"
Jul 24 23:29:46.869361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.869345    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d1032f1-7b09-452a-98ed-05f59629d0c8" containerName="main"
Jul 24 23:29:46.869361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.869359    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30f3b73-6432-499d-9517-98b5bb187982" containerName="main"
Jul 24 23:29:46.874317 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.874298    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:46.876916 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.876895    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:29:46.877256 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.877205    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:29:46.878707 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:46.878688    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"]
Jul 24 23:29:47.007041 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.007007    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b538dd-03cf-4045-a742-b30bbc921021-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb\" (UID: \"c8b538dd-03cf-4045-a742-b30bbc921021\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:47.108236 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.108210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b538dd-03cf-4045-a742-b30bbc921021-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb\" (UID: \"c8b538dd-03cf-4045-a742-b30bbc921021\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:47.108785 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.108767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b538dd-03cf-4045-a742-b30bbc921021-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb\" (UID: \"c8b538dd-03cf-4045-a742-b30bbc921021\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:47.186459 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.186395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:47.322170 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.322140    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"]
Jul 24 23:29:47.323933 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:29:47.323891    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b538dd_03cf_4045_a742_b30bbc921021.slice/crio-9612bf81dc670606161b565fc87c32f0faec62fe68cd57fa32a986f9f7d7073b WatchSource:0}: Error finding container 9612bf81dc670606161b565fc87c32f0faec62fe68cd57fa32a986f9f7d7073b: Status 404 returned error can't find the container with id 9612bf81dc670606161b565fc87c32f0faec62fe68cd57fa32a986f9f7d7073b
Jul 24 23:29:47.325827 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.325811    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:29:47.405440 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:47.405409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb" event={"ID":"c8b538dd-03cf-4045-a742-b30bbc921021","Type":"ContainerStarted","Data":"9612bf81dc670606161b565fc87c32f0faec62fe68cd57fa32a986f9f7d7073b"}
Jul 24 23:29:53.443854 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:53.443820    2564 generic.go:356] "Generic (PLEG): container finished" podID="c8b538dd-03cf-4045-a742-b30bbc921021" containerID="562cd09423c16d6286adb5e9b665ccac4e93cfe91d30f2d28b4bd0b15f34a389" exitCode=0
Jul 24 23:29:53.444239 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:53.443912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb" event={"ID":"c8b538dd-03cf-4045-a742-b30bbc921021","Type":"ContainerDied","Data":"562cd09423c16d6286adb5e9b665ccac4e93cfe91d30f2d28b4bd0b15f34a389"}
Jul 24 23:29:54.585089 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:54.585066    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:54.682666 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:54.682635    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b538dd-03cf-4045-a742-b30bbc921021-maas-api-service-ca\") pod \"c8b538dd-03cf-4045-a742-b30bbc921021\" (UID: \"c8b538dd-03cf-4045-a742-b30bbc921021\") "
Jul 24 23:29:54.682972 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:54.682951    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8b538dd-03cf-4045-a742-b30bbc921021-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8b538dd-03cf-4045-a742-b30bbc921021" (UID: "c8b538dd-03cf-4045-a742-b30bbc921021"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:29:54.783581 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:54.783495    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b538dd-03cf-4045-a742-b30bbc921021-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:29:55.457202 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:55.457177    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"
Jul 24 23:29:55.457491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:55.457220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb" event={"ID":"c8b538dd-03cf-4045-a742-b30bbc921021","Type":"ContainerDied","Data":"9612bf81dc670606161b565fc87c32f0faec62fe68cd57fa32a986f9f7d7073b"}
Jul 24 23:29:55.457491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:55.457266    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9612bf81dc670606161b565fc87c32f0faec62fe68cd57fa32a986f9f7d7073b"
Jul 24 23:29:57.416539 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:57.416510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.515310 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:29:57.515274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:00.301704 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:00.301675    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:30:01.943103 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:01.943072    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"]
Jul 24 23:30:01.944250 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:01.944212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:01.945449 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:01.945427    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0f3852-njnlb"]
Jul 24 23:30:03.694303 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:03.694269    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8b538dd-03cf-4045-a742-b30bbc921021" path="/var/lib/kubelet/pods/c8b538dd-03cf-4045-a742-b30bbc921021/volumes"
Jul 24 23:30:03.854479 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:03.854448    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:06.710574 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:06.710536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:07.985740 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:07.985704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.086043 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:08.086009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.277210 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:08.277097    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.547597 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:08.547503    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.842270 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:08.842235    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:09.176155 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:09.176032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:10.221541 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:10.221508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:11.851171 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:11.851081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:11.924581 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:11.924553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:11.995504 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:11.995473    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:12.406810 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:12.406781    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:12.537757 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:12.537729    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:12.645485 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:12.645440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:15.284402 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:15.284371    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:30:16.353513 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:16.353479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:16.821032 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:16.820943    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.161660 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:17.161629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.328747 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:17.328718    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:19.920970 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:19.920938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:20.021116 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:20.021083    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:20.121697 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:20.121615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:20.284439 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:20.284409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:20.885533 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:20.885498    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:20.975210 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:20.975174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:24.415542 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:24.415508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.491831 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:27.491786    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:30:32.485209 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.485159    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"]
Jul 24 23:30:32.485759 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.485739    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8b538dd-03cf-4045-a742-b30bbc921021" containerName="revoke-keys"
Jul 24 23:30:32.485846 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.485762    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8b538dd-03cf-4045-a742-b30bbc921021" containerName="revoke-keys"
Jul 24 23:30:32.485920 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.485907    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8b538dd-03cf-4045-a742-b30bbc921021" containerName="revoke-keys"
Jul 24 23:30:32.493260 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.493240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:32.496309 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.496284    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:30:32.497210 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.497185    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"]
Jul 24 23:30:32.497381 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.497201    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:30:32.603862 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.603835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54881cc-f2b8-4316-b597-53d6fe741343-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5d3442-zk65d\" (UID: \"c54881cc-f2b8-4316-b597-53d6fe741343\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:32.704632 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.704601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54881cc-f2b8-4316-b597-53d6fe741343-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5d3442-zk65d\" (UID: \"c54881cc-f2b8-4316-b597-53d6fe741343\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:32.705272 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.705252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54881cc-f2b8-4316-b597-53d6fe741343-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5d3442-zk65d\" (UID: \"c54881cc-f2b8-4316-b597-53d6fe741343\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:32.805356 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:32.805274    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:33.147898 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:33.147871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"]
Jul 24 23:30:33.149821 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:30:33.149794    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54881cc_f2b8_4316_b597_53d6fe741343.slice/crio-8a30400a637dbe6bf07e6f2f859322b5a960f8eb6efcf1a34774c75296fa3fab WatchSource:0}: Error finding container 8a30400a637dbe6bf07e6f2f859322b5a960f8eb6efcf1a34774c75296fa3fab: Status 404 returned error can't find the container with id 8a30400a637dbe6bf07e6f2f859322b5a960f8eb6efcf1a34774c75296fa3fab
Jul 24 23:30:33.685666 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:33.685632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d" event={"ID":"c54881cc-f2b8-4316-b597-53d6fe741343","Type":"ContainerStarted","Data":"2c10a83db4dc4a9855514ce40ffa86bd302ee89b52da15bb07ebf9cd2e8a3db5"}
Jul 24 23:30:33.686084 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:33.685684    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d" event={"ID":"c54881cc-f2b8-4316-b597-53d6fe741343","Type":"ContainerStarted","Data":"8a30400a637dbe6bf07e6f2f859322b5a960f8eb6efcf1a34774c75296fa3fab"}
Jul 24 23:30:33.701830 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:33.701776    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d" podStartSLOduration=1.701761199 podStartE2EDuration="1.701761199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:33.700580147 +0000 UTC m=+2294.553906682" watchObservedRunningTime="2026-07-24 23:30:33.701761199 +0000 UTC m=+2294.555087738"
Jul 24 23:30:34.693083 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:34.693049    2564 generic.go:356] "Generic (PLEG): container finished" podID="c54881cc-f2b8-4316-b597-53d6fe741343" containerID="2c10a83db4dc4a9855514ce40ffa86bd302ee89b52da15bb07ebf9cd2e8a3db5" exitCode=0
Jul 24 23:30:34.693476 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:34.693098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d" event={"ID":"c54881cc-f2b8-4316-b597-53d6fe741343","Type":"ContainerDied","Data":"2c10a83db4dc4a9855514ce40ffa86bd302ee89b52da15bb07ebf9cd2e8a3db5"}
Jul 24 23:30:35.825878 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:35.825851    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:35.935816 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:35.935792    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54881cc-f2b8-4316-b597-53d6fe741343-maas-api-service-ca\") pod \"c54881cc-f2b8-4316-b597-53d6fe741343\" (UID: \"c54881cc-f2b8-4316-b597-53d6fe741343\") "
Jul 24 23:30:35.936111 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:35.936086    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54881cc-f2b8-4316-b597-53d6fe741343-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c54881cc-f2b8-4316-b597-53d6fe741343" (UID: "c54881cc-f2b8-4316-b597-53d6fe741343"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:36.036771 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:36.036708    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54881cc-f2b8-4316-b597-53d6fe741343-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:30:36.704666 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:36.704643    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"
Jul 24 23:30:36.704845 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:36.704690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d" event={"ID":"c54881cc-f2b8-4316-b597-53d6fe741343","Type":"ContainerDied","Data":"8a30400a637dbe6bf07e6f2f859322b5a960f8eb6efcf1a34774c75296fa3fab"}
Jul 24 23:30:36.704845 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:36.704737    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a30400a637dbe6bf07e6f2f859322b5a960f8eb6efcf1a34774c75296fa3fab"
Jul 24 23:30:40.995245 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:40.995208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:47.825580 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:47.825545    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:47.891712 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:47.891683    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:47.955079 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:47.955052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:52.559912 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:52.559825    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"]
Jul 24 23:30:52.561072 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:52.561029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:52.562582 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:52.562559    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5d3442-zk65d"]
Jul 24 23:30:52.940335 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:52.940299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:53.590531 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:53.590499    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:30:53.695727 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:53.695695    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54881cc-f2b8-4316-b597-53d6fe741343" path="/var/lib/kubelet/pods/c54881cc-f2b8-4316-b597-53d6fe741343/volumes"
Jul 24 23:30:58.820090 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:58.820035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:58.920708 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:58.920676    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:59.130141 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.130089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:59.324370 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.324340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:59.439778 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.439710    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"]
Jul 24 23:30:59.440184 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.440168    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54881cc-f2b8-4316-b597-53d6fe741343" containerName="revoke-keys"
Jul 24 23:30:59.440184 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.440185    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54881cc-f2b8-4316-b597-53d6fe741343" containerName="revoke-keys"
Jul 24 23:30:59.440271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.440263    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54881cc-f2b8-4316-b597-53d6fe741343" containerName="revoke-keys"
Jul 24 23:30:59.443307 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.443291    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:30:59.445631 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.445613    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:30:59.445728 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.445695    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:30:59.450548 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.450494    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"]
Jul 24 23:30:59.524432 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.524409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b58ab190-7bfa-468c-a3c1-0e67ba601ca4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh\" (UID: \"b58ab190-7bfa-468c-a3c1-0e67ba601ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:30:59.625596 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.625566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b58ab190-7bfa-468c-a3c1-0e67ba601ca4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh\" (UID: \"b58ab190-7bfa-468c-a3c1-0e67ba601ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:30:59.626104 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.626087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b58ab190-7bfa-468c-a3c1-0e67ba601ca4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh\" (UID: \"b58ab190-7bfa-468c-a3c1-0e67ba601ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:30:59.757393 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:30:59.757325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:31:00.099554 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:00.099525    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"]
Jul 24 23:31:00.101223 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:31:00.101194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58ab190_7bfa_468c_a3c1_0e67ba601ca4.slice/crio-9e581d1de1f1af54f30e1d72801064454d6eee7c098f53fcc777f976b80db2db WatchSource:0}: Error finding container 9e581d1de1f1af54f30e1d72801064454d6eee7c098f53fcc777f976b80db2db: Status 404 returned error can't find the container with id 9e581d1de1f1af54f30e1d72801064454d6eee7c098f53fcc777f976b80db2db
Jul 24 23:31:00.843357 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:00.843324    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh" event={"ID":"b58ab190-7bfa-468c-a3c1-0e67ba601ca4","Type":"ContainerStarted","Data":"df1f3da96808df496f0be8a5d7237628fa4fead6f0357e90f40fb2051ff1d068"}
Jul 24 23:31:00.843527 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:00.843376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh" event={"ID":"b58ab190-7bfa-468c-a3c1-0e67ba601ca4","Type":"ContainerStarted","Data":"9e581d1de1f1af54f30e1d72801064454d6eee7c098f53fcc777f976b80db2db"}
Jul 24 23:31:00.860051 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:00.859999    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh" podStartSLOduration=1.859981686 podStartE2EDuration="1.859981686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:00.857793525 +0000 UTC m=+2321.711120064" watchObservedRunningTime="2026-07-24 23:31:00.859981686 +0000 UTC m=+2321.713308225"
Jul 24 23:31:01.850511 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:01.850476    2564 generic.go:356] "Generic (PLEG): container finished" podID="b58ab190-7bfa-468c-a3c1-0e67ba601ca4" containerID="df1f3da96808df496f0be8a5d7237628fa4fead6f0357e90f40fb2051ff1d068" exitCode=0
Jul 24 23:31:01.850511 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:01.850515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh" event={"ID":"b58ab190-7bfa-468c-a3c1-0e67ba601ca4","Type":"ContainerDied","Data":"df1f3da96808df496f0be8a5d7237628fa4fead6f0357e90f40fb2051ff1d068"}
Jul 24 23:31:02.998681 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:02.998659    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:31:03.054840 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:03.054813    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b58ab190-7bfa-468c-a3c1-0e67ba601ca4-maas-api-service-ca\") pod \"b58ab190-7bfa-468c-a3c1-0e67ba601ca4\" (UID: \"b58ab190-7bfa-468c-a3c1-0e67ba601ca4\") "
Jul 24 23:31:03.055199 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:03.055181    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b58ab190-7bfa-468c-a3c1-0e67ba601ca4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b58ab190-7bfa-468c-a3c1-0e67ba601ca4" (UID: "b58ab190-7bfa-468c-a3c1-0e67ba601ca4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:03.156074 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:03.156010    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b58ab190-7bfa-468c-a3c1-0e67ba601ca4-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:31:03.863776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:03.863750    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"
Jul 24 23:31:03.863776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:03.863756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh" event={"ID":"b58ab190-7bfa-468c-a3c1-0e67ba601ca4","Type":"ContainerDied","Data":"9e581d1de1f1af54f30e1d72801064454d6eee7c098f53fcc777f976b80db2db"}
Jul 24 23:31:03.863956 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:03.863802    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e581d1de1f1af54f30e1d72801064454d6eee7c098f53fcc777f976b80db2db"
Jul 24 23:31:08.621276 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:08.621242    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:08.682394 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:08.682365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:15.188958 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:15.188928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:15.288025 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:15.287991    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:15.388269 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:15.388231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:19.566574 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:19.566535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:19.576661 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:19.576635    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"]
Jul 24 23:31:19.580493 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:19.580471    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-183bb6-hfkrh"]
Jul 24 23:31:19.695795 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:19.695755    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b58ab190-7bfa-468c-a3c1-0e67ba601ca4" path="/var/lib/kubelet/pods/b58ab190-7bfa-468c-a3c1-0e67ba601ca4/volumes"
Jul 24 23:31:19.801097 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:19.801070    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:31:19.969904 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:19.969874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:26.068861 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:26.068833    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:26.169044 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:26.169009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:26.350914 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:26.350886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:26.568861 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:26.568831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:26.913024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:26.912980    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:27.239084 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:27.239005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:27.375814 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:27.375781    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:30.292672 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:30.292642    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:30.424249 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:30.424218    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:30.610499 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:30.610471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:30.743406 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:30.743380    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:31.163810 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:31.163778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:31.274880 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:31.274850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:34.813427 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:34.813396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:35.153008 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:35.152977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:35.459836 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:35.459767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:35.606325 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:35.606294    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:31:35.712878 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:35.712812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:38.270268 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:38.270233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:38.352835 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:38.352798    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:38.416507 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:38.416481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:38.655586 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:38.655535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:38.963906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:38.963828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:39.064304 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:39.064266    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:42.767588 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:42.767561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:43.703613 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:43.703586    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:31:52.300833 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.300802    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"]
Jul 24 23:31:52.301254 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.301236    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b58ab190-7bfa-468c-a3c1-0e67ba601ca4" containerName="revoke-keys"
Jul 24 23:31:52.301254 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.301247    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b58ab190-7bfa-468c-a3c1-0e67ba601ca4" containerName="revoke-keys"
Jul 24 23:31:52.301331 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.301323    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b58ab190-7bfa-468c-a3c1-0e67ba601ca4" containerName="revoke-keys"
Jul 24 23:31:52.304599 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.304582    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.309100 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.309077    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"auth-test-model-fe475f-kserve-self-signed-certs\""
Jul 24 23:31:52.310089 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.310064    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"kube-root-ca.crt\""
Jul 24 23:31:52.310089 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.310084    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"default-dockercfg-b6hrx\""
Jul 24 23:31:52.310283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.310071    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"openshift-service-ca.crt\""
Jul 24 23:31:52.321871 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.321847    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"]
Jul 24 23:31:52.343727 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.343706    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:52.378755 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.378729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-home\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.378863 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.378757    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-tmp-dir\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.378863 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.378775    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-dshm\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.378863 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.378801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b1624c-683b-474e-98d7-50ba08282bc8-tls-certs\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.378989 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.378869    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cq6c\" (UniqueName: \"kubernetes.io/projected/97b1624c-683b-474e-98d7-50ba08282bc8-kube-api-access-4cq6c\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.378989 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.378901    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-model-cache\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479337 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-home\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479480 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-tmp-dir\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479480 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-dshm\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479480 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b1624c-683b-474e-98d7-50ba08282bc8-tls-certs\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479480 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cq6c\" (UniqueName: \"kubernetes.io/projected/97b1624c-683b-474e-98d7-50ba08282bc8-kube-api-access-4cq6c\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479680 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479510    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-model-cache\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479680 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-home\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479848 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-model-cache\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.479980 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.479960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-tmp-dir\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.481643 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.481614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-dshm\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.481945 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.481929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b1624c-683b-474e-98d7-50ba08282bc8-tls-certs\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.487058 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.487030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cq6c\" (UniqueName: \"kubernetes.io/projected/97b1624c-683b-474e-98d7-50ba08282bc8-kube-api-access-4cq6c\") pod \"auth-test-model-fe475f-kserve-795d985f8c-q4hbs\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") " pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.613683 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.613654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:31:52.759135 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:52.759101    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"]
Jul 24 23:31:52.759676 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:31:52.759648    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97b1624c_683b_474e_98d7_50ba08282bc8.slice/crio-1d2d904deec81c382b87c2ef22a4a4103cffd58d09b8876a80f7bf03f4f4f428 WatchSource:0}: Error finding container 1d2d904deec81c382b87c2ef22a4a4103cffd58d09b8876a80f7bf03f4f4f428: Status 404 returned error can't find the container with id 1d2d904deec81c382b87c2ef22a4a4103cffd58d09b8876a80f7bf03f4f4f428
Jul 24 23:31:53.153564 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:53.153532    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs" event={"ID":"97b1624c-683b-474e-98d7-50ba08282bc8","Type":"ContainerStarted","Data":"4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d"}
Jul 24 23:31:53.153564 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:53.153577    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs" event={"ID":"97b1624c-683b-474e-98d7-50ba08282bc8","Type":"ContainerStarted","Data":"1d2d904deec81c382b87c2ef22a4a4103cffd58d09b8876a80f7bf03f4f4f428"}
Jul 24 23:31:53.174054 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:53.174005    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs" podStartSLOduration=1.173990668 podStartE2EDuration="1.173990668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:53.171058011 +0000 UTC m=+2374.024384550" watchObservedRunningTime="2026-07-24 23:31:53.173990668 +0000 UTC m=+2374.027317202"
Jul 24 23:31:56.093895 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:31:56.093857    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:32:02.614554 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:02.614512    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:32:02.615028 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:02.614568    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:32:02.627361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:02.627332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:32:03.223460 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:03.223430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:32:17.052666 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:17.052633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:17.153829 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:17.153800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:19.808509 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:19.808479    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:32:19.818034 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:19.818014    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:32:21.100203 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.100171    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"]
Jul 24 23:32:21.104093 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.104077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.107538 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.107518    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"kube-root-ca.crt\""
Jul 24 23:32:21.107538 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.107525    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"openshift-service-ca.crt\""
Jul 24 23:32:21.107748 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.107525    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"auth-test-model-6f3ce2-kserve-self-signed-certs\""
Jul 24 23:32:21.107748 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.107525    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"default-dockercfg-dclkw\""
Jul 24 23:32:21.113064 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.113044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"]
Jul 24 23:32:21.125407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.125381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:21.237014 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.236973    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-home\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.237224 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.237072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnr6n\" (UniqueName: \"kubernetes.io/projected/783e8eea-7d7a-4d87-bf41-dce2905387b3-kube-api-access-nnr6n\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.237224 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.237211    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-dshm\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.237342 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.237259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-tmp-dir\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.237342 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.237330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/783e8eea-7d7a-4d87-bf41-dce2905387b3-tls-certs\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.237417 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.237372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-model-cache\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338395 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnr6n\" (UniqueName: \"kubernetes.io/projected/783e8eea-7d7a-4d87-bf41-dce2905387b3-kube-api-access-nnr6n\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338566 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-dshm\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338566 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-tmp-dir\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338566 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/783e8eea-7d7a-4d87-bf41-dce2905387b3-tls-certs\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338566 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-model-cache\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338796 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-home\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.338952 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-tmp-dir\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.339038 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.338996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-model-cache\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.339096 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.339044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-home\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.340678 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.340657    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-dshm\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.341557 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.341535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/783e8eea-7d7a-4d87-bf41-dce2905387b3-tls-certs\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.346842 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.346811    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnr6n\" (UniqueName: \"kubernetes.io/projected/783e8eea-7d7a-4d87-bf41-dce2905387b3-kube-api-access-nnr6n\") pod \"auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") " pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.414228 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.414154    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:21.569704 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:21.569677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"]
Jul 24 23:32:21.570720 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:32:21.570684    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod783e8eea_7d7a_4d87_bf41_dce2905387b3.slice/crio-7aab691b83b25d9d5e90a7328cf7c67c241e5db289214a1f4c7a58a7baf01903 WatchSource:0}: Error finding container 7aab691b83b25d9d5e90a7328cf7c67c241e5db289214a1f4c7a58a7baf01903: Status 404 returned error can't find the container with id 7aab691b83b25d9d5e90a7328cf7c67c241e5db289214a1f4c7a58a7baf01903
Jul 24 23:32:22.320352 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:22.320313    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf" event={"ID":"783e8eea-7d7a-4d87-bf41-dce2905387b3","Type":"ContainerStarted","Data":"76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458"}
Jul 24 23:32:22.320352 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:22.320357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf" event={"ID":"783e8eea-7d7a-4d87-bf41-dce2905387b3","Type":"ContainerStarted","Data":"7aab691b83b25d9d5e90a7328cf7c67c241e5db289214a1f4c7a58a7baf01903"}
Jul 24 23:32:22.338701 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:22.338643    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf" podStartSLOduration=1.338624353 podStartE2EDuration="1.338624353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:32:22.338149466 +0000 UTC m=+2403.191476049" watchObservedRunningTime="2026-07-24 23:32:22.338624353 +0000 UTC m=+2403.191950894"
Jul 24 23:32:25.192598 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:25.192568    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:32:31.414877 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:31.414832    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:31.415394 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:31.414900    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:31.427454 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:31.427432    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:32.390639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:32.390609    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:32:39.507115 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:39.507067    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:32:57.987906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:32:57.987866    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:33:13.092825 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:33:13.092788    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:33:32.986382 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:33:32.986345    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:33:52.091555 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:33:52.091516    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:33:58.004832 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:33:58.004799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:08.092230 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:08.092193    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:34:22.781431 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:22.781350    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"]
Jul 24 23:34:22.806594 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:22.806557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:24.073867 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:24.073833    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"]
Jul 24 23:34:24.095898 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:24.095868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:27.000214 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:27.000167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:34:37.801472 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:37.801403    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs" podUID="97b1624c-683b-474e-98d7-50ba08282bc8" containerName="main" containerID="cri-o://4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d" gracePeriod=45
Jul 24 23:34:38.070454 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.070428    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:34:38.133852 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.133819    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-model-cache\") pod \"97b1624c-683b-474e-98d7-50ba08282bc8\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") "
Jul 24 23:34:38.134052 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.133898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-tmp-dir\") pod \"97b1624c-683b-474e-98d7-50ba08282bc8\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") "
Jul 24 23:34:38.134052 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.133976    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-home\") pod \"97b1624c-683b-474e-98d7-50ba08282bc8\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") "
Jul 24 23:34:38.134052 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134007    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cq6c\" (UniqueName: \"kubernetes.io/projected/97b1624c-683b-474e-98d7-50ba08282bc8-kube-api-access-4cq6c\") pod \"97b1624c-683b-474e-98d7-50ba08282bc8\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") "
Jul 24 23:34:38.134245 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134056    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-dshm\") pod \"97b1624c-683b-474e-98d7-50ba08282bc8\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") "
Jul 24 23:34:38.134245 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134082    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b1624c-683b-474e-98d7-50ba08282bc8-tls-certs\") pod \"97b1624c-683b-474e-98d7-50ba08282bc8\" (UID: \"97b1624c-683b-474e-98d7-50ba08282bc8\") "
Jul 24 23:34:38.134245 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134114    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-model-cache" (OuterVolumeSpecName: "model-cache") pod "97b1624c-683b-474e-98d7-50ba08282bc8" (UID: "97b1624c-683b-474e-98d7-50ba08282bc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:38.134245 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134198    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "97b1624c-683b-474e-98d7-50ba08282bc8" (UID: "97b1624c-683b-474e-98d7-50ba08282bc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:38.134477 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134392    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:38.134477 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134435    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:38.134589 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.134568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-home" (OuterVolumeSpecName: "home") pod "97b1624c-683b-474e-98d7-50ba08282bc8" (UID: "97b1624c-683b-474e-98d7-50ba08282bc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:38.136774 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.136747    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b1624c-683b-474e-98d7-50ba08282bc8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "97b1624c-683b-474e-98d7-50ba08282bc8" (UID: "97b1624c-683b-474e-98d7-50ba08282bc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:34:38.136906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.136791    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-dshm" (OuterVolumeSpecName: "dshm") pod "97b1624c-683b-474e-98d7-50ba08282bc8" (UID: "97b1624c-683b-474e-98d7-50ba08282bc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:38.136906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.136796    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b1624c-683b-474e-98d7-50ba08282bc8-kube-api-access-4cq6c" (OuterVolumeSpecName: "kube-api-access-4cq6c") pod "97b1624c-683b-474e-98d7-50ba08282bc8" (UID: "97b1624c-683b-474e-98d7-50ba08282bc8"). InnerVolumeSpecName "kube-api-access-4cq6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:34:38.152974 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.152943    2564 generic.go:356] "Generic (PLEG): container finished" podID="97b1624c-683b-474e-98d7-50ba08282bc8" containerID="4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d" exitCode=0
Jul 24 23:34:38.153090 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.153003    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"
Jul 24 23:34:38.153090 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.153004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs" event={"ID":"97b1624c-683b-474e-98d7-50ba08282bc8","Type":"ContainerDied","Data":"4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d"}
Jul 24 23:34:38.153212 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.153116    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs" event={"ID":"97b1624c-683b-474e-98d7-50ba08282bc8","Type":"ContainerDied","Data":"1d2d904deec81c382b87c2ef22a4a4103cffd58d09b8876a80f7bf03f4f4f428"}
Jul 24 23:34:38.153212 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.153154    2564 scope.go:117] "RemoveContainer" containerID="4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d"
Jul 24 23:34:38.163340 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.163262    2564 scope.go:117] "RemoveContainer" containerID="4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d"
Jul 24 23:34:38.165250 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:34:38.165230    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d\": container with ID starting with 4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d not found: ID does not exist" containerID="4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d"
Jul 24 23:34:38.165345 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.165262    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d"} err="failed to get container status \"4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d\": rpc error: code = NotFound desc = could not find container \"4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d\": container with ID starting with 4c91cbf0c0ba17036886fed99cec3ab58380ba6b6a963c720d79c9328731093d not found: ID does not exist"
Jul 24 23:34:38.174339 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.174313    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"]
Jul 24 23:34:38.176698 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.176679    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/auth-test-model-fe475f-kserve-795d985f8c-q4hbs"]
Jul 24 23:34:38.235043 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.235003    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:38.235043 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.235044    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cq6c\" (UniqueName: \"kubernetes.io/projected/97b1624c-683b-474e-98d7-50ba08282bc8-kube-api-access-4cq6c\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:38.235283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.235060    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97b1624c-683b-474e-98d7-50ba08282bc8-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:38.235283 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:38.235075    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97b1624c-683b-474e-98d7-50ba08282bc8-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.093272 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.093237    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf" podUID="783e8eea-7d7a-4d87-bf41-dce2905387b3" containerName="main" containerID="cri-o://76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458" gracePeriod=45
Jul 24 23:34:39.356416 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.356353    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:34:39.448769 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.448733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-home\") pod \"783e8eea-7d7a-4d87-bf41-dce2905387b3\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") "
Jul 24 23:34:39.448942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.448783    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-dshm\") pod \"783e8eea-7d7a-4d87-bf41-dce2905387b3\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") "
Jul 24 23:34:39.448942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.448826    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-model-cache\") pod \"783e8eea-7d7a-4d87-bf41-dce2905387b3\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") "
Jul 24 23:34:39.448942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.448898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-tmp-dir\") pod \"783e8eea-7d7a-4d87-bf41-dce2905387b3\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") "
Jul 24 23:34:39.448942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.448930    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnr6n\" (UniqueName: \"kubernetes.io/projected/783e8eea-7d7a-4d87-bf41-dce2905387b3-kube-api-access-nnr6n\") pod \"783e8eea-7d7a-4d87-bf41-dce2905387b3\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") "
Jul 24 23:34:39.449200 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.448996    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/783e8eea-7d7a-4d87-bf41-dce2905387b3-tls-certs\") pod \"783e8eea-7d7a-4d87-bf41-dce2905387b3\" (UID: \"783e8eea-7d7a-4d87-bf41-dce2905387b3\") "
Jul 24 23:34:39.449200 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.449041    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-home" (OuterVolumeSpecName: "home") pod "783e8eea-7d7a-4d87-bf41-dce2905387b3" (UID: "783e8eea-7d7a-4d87-bf41-dce2905387b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:39.449200 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.449171    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "783e8eea-7d7a-4d87-bf41-dce2905387b3" (UID: "783e8eea-7d7a-4d87-bf41-dce2905387b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:39.449360 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.449187    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "783e8eea-7d7a-4d87-bf41-dce2905387b3" (UID: "783e8eea-7d7a-4d87-bf41-dce2905387b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:39.449360 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.449352    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.449461 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.449369    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.449461 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.449377    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.451215 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.451193    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-dshm" (OuterVolumeSpecName: "dshm") pod "783e8eea-7d7a-4d87-bf41-dce2905387b3" (UID: "783e8eea-7d7a-4d87-bf41-dce2905387b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:34:39.451334 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.451298    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783e8eea-7d7a-4d87-bf41-dce2905387b3-kube-api-access-nnr6n" (OuterVolumeSpecName: "kube-api-access-nnr6n") pod "783e8eea-7d7a-4d87-bf41-dce2905387b3" (UID: "783e8eea-7d7a-4d87-bf41-dce2905387b3"). InnerVolumeSpecName "kube-api-access-nnr6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:34:39.451384 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.451329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/783e8eea-7d7a-4d87-bf41-dce2905387b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "783e8eea-7d7a-4d87-bf41-dce2905387b3" (UID: "783e8eea-7d7a-4d87-bf41-dce2905387b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:34:39.550040 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.550002    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/783e8eea-7d7a-4d87-bf41-dce2905387b3-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.550040 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.550034    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnr6n\" (UniqueName: \"kubernetes.io/projected/783e8eea-7d7a-4d87-bf41-dce2905387b3-kube-api-access-nnr6n\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.550040 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.550049    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/783e8eea-7d7a-4d87-bf41-dce2905387b3-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:34:39.695773 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:39.695692    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b1624c-683b-474e-98d7-50ba08282bc8" path="/var/lib/kubelet/pods/97b1624c-683b-474e-98d7-50ba08282bc8/volumes"
Jul 24 23:34:40.167174 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.167097    2564 generic.go:356] "Generic (PLEG): container finished" podID="783e8eea-7d7a-4d87-bf41-dce2905387b3" containerID="76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458" exitCode=0
Jul 24 23:34:40.167174 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.167157    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"
Jul 24 23:34:40.167694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.167192    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf" event={"ID":"783e8eea-7d7a-4d87-bf41-dce2905387b3","Type":"ContainerDied","Data":"76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458"}
Jul 24 23:34:40.167694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.167236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf" event={"ID":"783e8eea-7d7a-4d87-bf41-dce2905387b3","Type":"ContainerDied","Data":"7aab691b83b25d9d5e90a7328cf7c67c241e5db289214a1f4c7a58a7baf01903"}
Jul 24 23:34:40.167694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.167254    2564 scope.go:117] "RemoveContainer" containerID="76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458"
Jul 24 23:34:40.176487 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.176471    2564 scope.go:117] "RemoveContainer" containerID="76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458"
Jul 24 23:34:40.176782 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:34:40.176751    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458\": container with ID starting with 76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458 not found: ID does not exist" containerID="76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458"
Jul 24 23:34:40.176839 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.176784    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458"} err="failed to get container status \"76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458\": rpc error: code = NotFound desc = could not find container \"76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458\": container with ID starting with 76b9d12899a651713c221cfbd4559e08cdb08077c4a436cd2469e4d4632b3458 not found: ID does not exist"
Jul 24 23:34:40.184881 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.184856    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"]
Jul 24 23:34:40.188342 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:40.188319    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/auth-test-model-6f3ce2-kserve-fdf8ddf47-w7hhf"]
Jul 24 23:34:41.695593 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:41.695552    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="783e8eea-7d7a-4d87-bf41-dce2905387b3" path="/var/lib/kubelet/pods/783e8eea-7d7a-4d87-bf41-dce2905387b3/volumes"
Jul 24 23:34:45.698807 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.698772    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"]
Jul 24 23:34:45.699361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.699346    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b1624c-683b-474e-98d7-50ba08282bc8" containerName="main"
Jul 24 23:34:45.699361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.699363    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b1624c-683b-474e-98d7-50ba08282bc8" containerName="main"
Jul 24 23:34:45.699463 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.699394    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="783e8eea-7d7a-4d87-bf41-dce2905387b3" containerName="main"
Jul 24 23:34:45.699463 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.699400    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="783e8eea-7d7a-4d87-bf41-dce2905387b3" containerName="main"
Jul 24 23:34:45.699536 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.699476    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="783e8eea-7d7a-4d87-bf41-dce2905387b3" containerName="main"
Jul 24 23:34:45.699536 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.699484    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b1624c-683b-474e-98d7-50ba08282bc8" containerName="main"
Jul 24 23:34:45.703804 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.703789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.706308 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.706285    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"sub-test-model-fe475f-kserve-self-signed-certs\""
Jul 24 23:34:45.707552 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.707519    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"kube-root-ca.crt\""
Jul 24 23:34:45.707904 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.707841    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"default-dockercfg-b6hrx\""
Jul 24 23:34:45.708015 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.707926    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"openshift-service-ca.crt\""
Jul 24 23:34:45.715960 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.715937    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"]
Jul 24 23:34:45.722497 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.722464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:45.807740 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.807706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-dshm\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.807740 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.807735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-model-cache\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.807982 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.807758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-home\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.807982 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.807868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkr79\" (UniqueName: \"kubernetes.io/projected/f5e15227-29b1-4afa-9b14-0fa3184725aa-kube-api-access-kkr79\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.807982 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.807959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e15227-29b1-4afa-9b14-0fa3184725aa-tls-certs\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.808182 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.808051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-tmp-dir\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.908724 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.908689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-tmp-dir\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.908724 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.908731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-dshm\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.908748    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-model-cache\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.908766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-home\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.908803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkr79\" (UniqueName: \"kubernetes.io/projected/f5e15227-29b1-4afa-9b14-0fa3184725aa-kube-api-access-kkr79\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.908845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e15227-29b1-4afa-9b14-0fa3184725aa-tls-certs\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909276 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.909224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-home\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909276 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.909229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-tmp-dir\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.909355 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.909269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-model-cache\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.911087 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.911058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-dshm\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.911424 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.911404    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e15227-29b1-4afa-9b14-0fa3184725aa-tls-certs\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:45.917454 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:45.917432    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkr79\" (UniqueName: \"kubernetes.io/projected/f5e15227-29b1-4afa-9b14-0fa3184725aa-kube-api-access-kkr79\") pod \"sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") " pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:46.016566 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:46.016478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:46.158581 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:46.158545    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"]
Jul 24 23:34:46.159547 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:34:46.159520    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5e15227_29b1_4afa_9b14_0fa3184725aa.slice/crio-84671b6192fe433b0557f61d71b358a615dca89835e318f7585bd1977175419a WatchSource:0}: Error finding container 84671b6192fe433b0557f61d71b358a615dca89835e318f7585bd1977175419a: Status 404 returned error can't find the container with id 84671b6192fe433b0557f61d71b358a615dca89835e318f7585bd1977175419a
Jul 24 23:34:46.203693 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:46.203657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v" event={"ID":"f5e15227-29b1-4afa-9b14-0fa3184725aa","Type":"ContainerStarted","Data":"84671b6192fe433b0557f61d71b358a615dca89835e318f7585bd1977175419a"}
Jul 24 23:34:46.692705 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:46.692669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:47.210522 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:47.210486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v" event={"ID":"f5e15227-29b1-4afa-9b14-0fa3184725aa","Type":"ContainerStarted","Data":"48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27"}
Jul 24 23:34:47.232020 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:47.231972    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v" podStartSLOduration=2.231956485 podStartE2EDuration="2.231956485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:47.230327772 +0000 UTC m=+2548.083654312" watchObservedRunningTime="2026-07-24 23:34:47.231956485 +0000 UTC m=+2548.085283025"
Jul 24 23:34:49.687612 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:49.687573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:34:56.016831 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:56.016791    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:56.017376 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:56.016849    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:56.029404 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:56.029371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:34:56.277157 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:34:56.277062    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:35:07.999381 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:07.999343    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"]
Jul 24 23:35:08.003193 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.003170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.005994 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.005973    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"kube-root-ca.crt\""
Jul 24 23:35:08.006545 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvs9c\" (UniqueName: \"kubernetes.io/projected/5ea04701-cb31-4cf2-83ad-2e19deb3690c-kube-api-access-vvs9c\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.006645 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006569    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-home\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.006694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-model-cache\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.006731 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tls-certs\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.006786 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006736    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tmp-dir\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.006786 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-dshm\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.006892 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.006874    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"openshift-service-ca.crt\""
Jul 24 23:35:08.007172 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.007152    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"sub-test-model-6f3ce2-kserve-self-signed-certs\""
Jul 24 23:35:08.007272 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.007214    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"default-dockercfg-dclkw\""
Jul 24 23:35:08.011802 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.011779    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"]
Jul 24 23:35:08.039359 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.039328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:08.107410 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tmp-dir\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107410 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-dshm\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107647 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvs9c\" (UniqueName: \"kubernetes.io/projected/5ea04701-cb31-4cf2-83ad-2e19deb3690c-kube-api-access-vvs9c\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107647 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-home\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107647 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107554    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-model-cache\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107647 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tls-certs\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107887 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tmp-dir\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107947 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-home\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.107947 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.107934    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-model-cache\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.109672 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.109654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-dshm\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.110048 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.110029    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tls-certs\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.115484 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.115465    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvs9c\" (UniqueName: \"kubernetes.io/projected/5ea04701-cb31-4cf2-83ad-2e19deb3690c-kube-api-access-vvs9c\") pod \"sub-test-model-6f3ce2-kserve-547849d4f-6gmp5\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") " pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.315985 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.315876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:08.452437 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:08.452406    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"]
Jul 24 23:35:08.454228 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:35:08.454205    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ea04701_cb31_4cf2_83ad_2e19deb3690c.slice/crio-122e24c64de23ac147c1c9e59e74a8c777c1544e4e5c1c3d1a68b8f6a2af398c WatchSource:0}: Error finding container 122e24c64de23ac147c1c9e59e74a8c777c1544e4e5c1c3d1a68b8f6a2af398c: Status 404 returned error can't find the container with id 122e24c64de23ac147c1c9e59e74a8c777c1544e4e5c1c3d1a68b8f6a2af398c
Jul 24 23:35:09.351704 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:09.351665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5" event={"ID":"5ea04701-cb31-4cf2-83ad-2e19deb3690c","Type":"ContainerStarted","Data":"1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8"}
Jul 24 23:35:09.352110 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:09.351720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5" event={"ID":"5ea04701-cb31-4cf2-83ad-2e19deb3690c","Type":"ContainerStarted","Data":"122e24c64de23ac147c1c9e59e74a8c777c1544e4e5c1c3d1a68b8f6a2af398c"}
Jul 24 23:35:09.369556 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:09.369501    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5" podStartSLOduration=2.369483812 podStartE2EDuration="2.369483812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:35:09.366921792 +0000 UTC m=+2570.220248353" watchObservedRunningTime="2026-07-24 23:35:09.369483812 +0000 UTC m=+2570.222810352"
Jul 24 23:35:11.990362 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:11.990317    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:35:18.316153 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:18.316103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:18.316554 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:18.316171    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:18.328731 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:18.328706    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:18.417645 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:18.417617    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:35:26.391974 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:26.391923    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:35:46.974779 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:46.974738    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"]
Jul 24 23:35:46.988489 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:46.988462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:48.177834 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:48.177797    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"]
Jul 24 23:35:48.225364 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:48.225333    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:49.584136 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:49.584089    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:35:58.553232 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.553198    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"]
Jul 24 23:35:58.558188 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.558159    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.561696 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.561663    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe475f\"/\"rate-test-model-fe475f-kserve-self-signed-certs\""
Jul 24 23:35:58.577493 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.577367    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"]
Jul 24 23:35:58.581792 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.581764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:58.584823 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.584799    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-model-cache\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.584929 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.584839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzjpv\" (UniqueName: \"kubernetes.io/projected/fe30c4d8-8ef7-4777-af04-a7657990d05b-kube-api-access-vzjpv\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.584929 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.584861    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe30c4d8-8ef7-4777-af04-a7657990d05b-tls-certs\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.585015 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.584927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-tmp-dir\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.585051 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.585032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-home\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.585085 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.585051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-dshm\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686114 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe30c4d8-8ef7-4777-af04-a7657990d05b-tls-certs\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686352 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-tmp-dir\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686352 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-home\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686352 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-dshm\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686352 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686342    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-model-cache\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686568 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzjpv\" (UniqueName: \"kubernetes.io/projected/fe30c4d8-8ef7-4777-af04-a7657990d05b-kube-api-access-vzjpv\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686652 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686625    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-tmp-dir\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686717 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-home\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.686717 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.686689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-model-cache\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.688459 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.688440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-dshm\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.688704 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.688687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe30c4d8-8ef7-4777-af04-a7657990d05b-tls-certs\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.695740 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.695719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzjpv\" (UniqueName: \"kubernetes.io/projected/fe30c4d8-8ef7-4777-af04-a7657990d05b-kube-api-access-vzjpv\") pod \"rate-test-model-fe475f-kserve-788df45c9f-6n8hm\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") " pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:58.870307 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:58.870260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:35:59.017392 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:59.017361    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"]
Jul 24 23:35:59.018781 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:35:59.018756    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe30c4d8_8ef7_4777_af04_a7657990d05b.slice/crio-49e08d60fe5376439c28229dcdfb584215e5d2b37dacf2f493fa4d637b37e2f2 WatchSource:0}: Error finding container 49e08d60fe5376439c28229dcdfb584215e5d2b37dacf2f493fa4d637b37e2f2: Status 404 returned error can't find the container with id 49e08d60fe5376439c28229dcdfb584215e5d2b37dacf2f493fa4d637b37e2f2
Jul 24 23:35:59.662813 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:59.662778    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm" event={"ID":"fe30c4d8-8ef7-4777-af04-a7657990d05b","Type":"ContainerStarted","Data":"7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285"}
Jul 24 23:35:59.662813 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:59.662815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm" event={"ID":"fe30c4d8-8ef7-4777-af04-a7657990d05b","Type":"ContainerStarted","Data":"49e08d60fe5376439c28229dcdfb584215e5d2b37dacf2f493fa4d637b37e2f2"}
Jul 24 23:35:59.681810 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:35:59.681759    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm" podStartSLOduration=1.681741745 podStartE2EDuration="1.681741745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:35:59.679429789 +0000 UTC m=+2620.532756330" watchObservedRunningTime="2026-07-24 23:35:59.681741745 +0000 UTC m=+2620.535068285"
Jul 24 23:36:01.099382 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:01.099340    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:36:01.994987 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:01.994937    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v" podUID="f5e15227-29b1-4afa-9b14-0fa3184725aa" containerName="main" containerID="cri-o://48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27" gracePeriod=45
Jul 24 23:36:02.254594 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.254529    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:36:02.319938 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.319908    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-model-cache\") pod \"f5e15227-29b1-4afa-9b14-0fa3184725aa\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") "
Jul 24 23:36:02.320154 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.319969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-home\") pod \"f5e15227-29b1-4afa-9b14-0fa3184725aa\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") "
Jul 24 23:36:02.320154 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320005    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-dshm\") pod \"f5e15227-29b1-4afa-9b14-0fa3184725aa\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") "
Jul 24 23:36:02.320154 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320024    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkr79\" (UniqueName: \"kubernetes.io/projected/f5e15227-29b1-4afa-9b14-0fa3184725aa-kube-api-access-kkr79\") pod \"f5e15227-29b1-4afa-9b14-0fa3184725aa\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") "
Jul 24 23:36:02.320154 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320074    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e15227-29b1-4afa-9b14-0fa3184725aa-tls-certs\") pod \"f5e15227-29b1-4afa-9b14-0fa3184725aa\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") "
Jul 24 23:36:02.320358 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320197    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-tmp-dir\") pod \"f5e15227-29b1-4afa-9b14-0fa3184725aa\" (UID: \"f5e15227-29b1-4afa-9b14-0fa3184725aa\") "
Jul 24 23:36:02.320358 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320243    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "f5e15227-29b1-4afa-9b14-0fa3184725aa" (UID: "f5e15227-29b1-4afa-9b14-0fa3184725aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:02.320358 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320318    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-home" (OuterVolumeSpecName: "home") pod "f5e15227-29b1-4afa-9b14-0fa3184725aa" (UID: "f5e15227-29b1-4afa-9b14-0fa3184725aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:02.320525 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320470    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f5e15227-29b1-4afa-9b14-0fa3184725aa" (UID: "f5e15227-29b1-4afa-9b14-0fa3184725aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:02.320615 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320597    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:02.320675 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320621    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:02.320675 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.320635    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:02.322265 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.322236    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-dshm" (OuterVolumeSpecName: "dshm") pod "f5e15227-29b1-4afa-9b14-0fa3184725aa" (UID: "f5e15227-29b1-4afa-9b14-0fa3184725aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:02.322367 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.322266    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e15227-29b1-4afa-9b14-0fa3184725aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f5e15227-29b1-4afa-9b14-0fa3184725aa" (UID: "f5e15227-29b1-4afa-9b14-0fa3184725aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:36:02.322367 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.322303    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e15227-29b1-4afa-9b14-0fa3184725aa-kube-api-access-kkr79" (OuterVolumeSpecName: "kube-api-access-kkr79") pod "f5e15227-29b1-4afa-9b14-0fa3184725aa" (UID: "f5e15227-29b1-4afa-9b14-0fa3184725aa"). InnerVolumeSpecName "kube-api-access-kkr79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:36:02.421837 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.421803    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e15227-29b1-4afa-9b14-0fa3184725aa-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:02.421837 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.421832    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkr79\" (UniqueName: \"kubernetes.io/projected/f5e15227-29b1-4afa-9b14-0fa3184725aa-kube-api-access-kkr79\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:02.421837 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.421842    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e15227-29b1-4afa-9b14-0fa3184725aa-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:02.682315 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.682278    2564 generic.go:356] "Generic (PLEG): container finished" podID="f5e15227-29b1-4afa-9b14-0fa3184725aa" containerID="48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27" exitCode=0
Jul 24 23:36:02.682491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.682323    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"
Jul 24 23:36:02.682491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.682358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v" event={"ID":"f5e15227-29b1-4afa-9b14-0fa3184725aa","Type":"ContainerDied","Data":"48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27"}
Jul 24 23:36:02.682491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.682402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v" event={"ID":"f5e15227-29b1-4afa-9b14-0fa3184725aa","Type":"ContainerDied","Data":"84671b6192fe433b0557f61d71b358a615dca89835e318f7585bd1977175419a"}
Jul 24 23:36:02.682491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.682422    2564 scope.go:117] "RemoveContainer" containerID="48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27"
Jul 24 23:36:02.692220 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.692202    2564 scope.go:117] "RemoveContainer" containerID="48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27"
Jul 24 23:36:02.692480 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:36:02.692461    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27\": container with ID starting with 48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27 not found: ID does not exist" containerID="48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27"
Jul 24 23:36:02.692551 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.692494    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27"} err="failed to get container status \"48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27\": rpc error: code = NotFound desc = could not find container \"48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27\": container with ID starting with 48f78cb0083ba9626d786b05824f61cf5fe62c1b5bf626c16bb4e5cc717a0d27 not found: ID does not exist"
Jul 24 23:36:02.704675 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.704648    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"]
Jul 24 23:36:02.709995 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:02.709966    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/sub-test-model-fe475f-kserve-594cc5d6b9-vlv8v"]
Jul 24 23:36:03.198368 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.198319    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5" podUID="5ea04701-cb31-4cf2-83ad-2e19deb3690c" containerName="main" containerID="cri-o://1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8" gracePeriod=45
Jul 24 23:36:03.454879 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.454813    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:36:03.531795 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.531757    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-model-cache\") pod \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") "
Jul 24 23:36:03.531951 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.531804    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tmp-dir\") pod \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") "
Jul 24 23:36:03.531951 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.531841    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvs9c\" (UniqueName: \"kubernetes.io/projected/5ea04701-cb31-4cf2-83ad-2e19deb3690c-kube-api-access-vvs9c\") pod \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") "
Jul 24 23:36:03.531951 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.531949    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tls-certs\") pod \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") "
Jul 24 23:36:03.532172 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.531964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-dshm\") pod \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") "
Jul 24 23:36:03.532172 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.532015    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-home\") pod \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\" (UID: \"5ea04701-cb31-4cf2-83ad-2e19deb3690c\") "
Jul 24 23:36:03.532172 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.532082    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ea04701-cb31-4cf2-83ad-2e19deb3690c" (UID: "5ea04701-cb31-4cf2-83ad-2e19deb3690c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:03.532172 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.532090    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ea04701-cb31-4cf2-83ad-2e19deb3690c" (UID: "5ea04701-cb31-4cf2-83ad-2e19deb3690c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:03.532399 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.532339    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:03.532399 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.532359    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:03.532399 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.532379    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-home" (OuterVolumeSpecName: "home") pod "5ea04701-cb31-4cf2-83ad-2e19deb3690c" (UID: "5ea04701-cb31-4cf2-83ad-2e19deb3690c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:03.534403 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.534375    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea04701-cb31-4cf2-83ad-2e19deb3690c-kube-api-access-vvs9c" (OuterVolumeSpecName: "kube-api-access-vvs9c") pod "5ea04701-cb31-4cf2-83ad-2e19deb3690c" (UID: "5ea04701-cb31-4cf2-83ad-2e19deb3690c"). InnerVolumeSpecName "kube-api-access-vvs9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:36:03.534646 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.534619    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ea04701-cb31-4cf2-83ad-2e19deb3690c" (UID: "5ea04701-cb31-4cf2-83ad-2e19deb3690c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:36:03.534757 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.534619    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-dshm" (OuterVolumeSpecName: "dshm") pod "5ea04701-cb31-4cf2-83ad-2e19deb3690c" (UID: "5ea04701-cb31-4cf2-83ad-2e19deb3690c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:36:03.633764 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.633724    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvs9c\" (UniqueName: \"kubernetes.io/projected/5ea04701-cb31-4cf2-83ad-2e19deb3690c-kube-api-access-vvs9c\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:03.633953 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.633770    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ea04701-cb31-4cf2-83ad-2e19deb3690c-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:03.633953 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.633785    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:03.633953 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.633797    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ea04701-cb31-4cf2-83ad-2e19deb3690c-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:36:03.693056 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.693027    2564 generic.go:356] "Generic (PLEG): container finished" podID="5ea04701-cb31-4cf2-83ad-2e19deb3690c" containerID="1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8" exitCode=0
Jul 24 23:36:03.693415 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.693107    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"
Jul 24 23:36:03.695794 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.695770    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e15227-29b1-4afa-9b14-0fa3184725aa" path="/var/lib/kubelet/pods/f5e15227-29b1-4afa-9b14-0fa3184725aa/volumes"
Jul 24 23:36:03.696191 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.696173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5" event={"ID":"5ea04701-cb31-4cf2-83ad-2e19deb3690c","Type":"ContainerDied","Data":"1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8"}
Jul 24 23:36:03.696237 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.696203    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5" event={"ID":"5ea04701-cb31-4cf2-83ad-2e19deb3690c","Type":"ContainerDied","Data":"122e24c64de23ac147c1c9e59e74a8c777c1544e4e5c1c3d1a68b8f6a2af398c"}
Jul 24 23:36:03.696237 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.696221    2564 scope.go:117] "RemoveContainer" containerID="1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8"
Jul 24 23:36:03.706342 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.706323    2564 scope.go:117] "RemoveContainer" containerID="1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8"
Jul 24 23:36:03.706589 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:36:03.706571    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8\": container with ID starting with 1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8 not found: ID does not exist" containerID="1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8"
Jul 24 23:36:03.706630 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.706598    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8"} err="failed to get container status \"1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8\": rpc error: code = NotFound desc = could not find container \"1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8\": container with ID starting with 1def4b05fce077d448ee0b932c89d401dd65f354c636fa0c713ea2cbb28bf7f8 not found: ID does not exist"
Jul 24 23:36:03.717725 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.717697    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"]
Jul 24 23:36:03.721612 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:03.721592    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/sub-test-model-6f3ce2-kserve-547849d4f-6gmp5"]
Jul 24 23:36:05.498236 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:05.498195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:05.696107 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:05.696069    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea04701-cb31-4cf2-83ad-2e19deb3690c" path="/var/lib/kubelet/pods/5ea04701-cb31-4cf2-83ad-2e19deb3690c/volumes"
Jul 24 23:36:08.871304 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:08.871256    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:36:08.871796 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:08.871320    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:36:08.884321 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:08.884285    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:36:09.742636 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:09.742609    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:36:17.399316 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.399274    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"]
Jul 24 23:36:17.399913 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.399892    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea04701-cb31-4cf2-83ad-2e19deb3690c" containerName="main"
Jul 24 23:36:17.400004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.399916    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea04701-cb31-4cf2-83ad-2e19deb3690c" containerName="main"
Jul 24 23:36:17.400004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.399944    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e15227-29b1-4afa-9b14-0fa3184725aa" containerName="main"
Jul 24 23:36:17.400004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.399953    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e15227-29b1-4afa-9b14-0fa3184725aa" containerName="main"
Jul 24 23:36:17.400187 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.400063    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea04701-cb31-4cf2-83ad-2e19deb3690c" containerName="main"
Jul 24 23:36:17.400187 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.400076    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e15227-29b1-4afa-9b14-0fa3184725aa" containerName="main"
Jul 24 23:36:17.404747 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.404725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.407672 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.407645    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"kube-root-ca.crt\""
Jul 24 23:36:17.410694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.410489    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"openshift-service-ca.crt\""
Jul 24 23:36:17.410694 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.410531    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"rate-test-model-6f3ce2-kserve-self-signed-certs\""
Jul 24 23:36:17.410889 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.410779    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6f3ce2\"/\"default-dockercfg-dclkw\""
Jul 24 23:36:17.412734 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.412712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"]
Jul 24 23:36:17.425117 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.425091    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:17.466413 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.466372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-dshm\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.466604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.466424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-home\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.466604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.466451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-model-cache\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.466604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.466512    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5th8\" (UniqueName: \"kubernetes.io/projected/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-kube-api-access-m5th8\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.466604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.466588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tmp-dir\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.466747 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.466631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tls-certs\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.567893 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.567856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tls-certs\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.567893 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.567901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-dshm\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568169 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.567930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-home\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568169 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.567948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-model-cache\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568169 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.567980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5th8\" (UniqueName: \"kubernetes.io/projected/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-kube-api-access-m5th8\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568169 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.568030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tmp-dir\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568442 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.568417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-home\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568518 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.568461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tmp-dir\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.568518 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.568490    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-model-cache\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.570263 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.570244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-dshm\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.570562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.570545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tls-certs\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.576441 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.576418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5th8\" (UniqueName: \"kubernetes.io/projected/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-kube-api-access-m5th8\") pod \"rate-test-model-6f3ce2-kserve-77677476b7-8d6jd\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") " pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.718780 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.718679    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:17.861164 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:17.861135    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"]
Jul 24 23:36:17.863585 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:36:17.863555    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08a8f1e7_d7d6_443a_8c08_1947af9d38ae.slice/crio-c57082e7ca3a9d699de934d27e98053fb879fb906002052b7f340facfd05d6ea WatchSource:0}: Error finding container c57082e7ca3a9d699de934d27e98053fb879fb906002052b7f340facfd05d6ea: Status 404 returned error can't find the container with id c57082e7ca3a9d699de934d27e98053fb879fb906002052b7f340facfd05d6ea
Jul 24 23:36:18.790507 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:18.790469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd" event={"ID":"08a8f1e7-d7d6-443a-8c08-1947af9d38ae","Type":"ContainerStarted","Data":"178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4"}
Jul 24 23:36:18.791021 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:18.790516    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd" event={"ID":"08a8f1e7-d7d6-443a-8c08-1947af9d38ae","Type":"ContainerStarted","Data":"c57082e7ca3a9d699de934d27e98053fb879fb906002052b7f340facfd05d6ea"}
Jul 24 23:36:18.807734 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:18.807678    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd" podStartSLOduration=1.80766067 podStartE2EDuration="1.80766067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:36:18.806076064 +0000 UTC m=+2639.659402604" watchObservedRunningTime="2026-07-24 23:36:18.80766067 +0000 UTC m=+2639.660987210"
Jul 24 23:36:19.932988 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:19.932951    2564 scope.go:117] "RemoveContainer" containerID="562cd09423c16d6286adb5e9b665ccac4e93cfe91d30f2d28b4bd0b15f34a389"
Jul 24 23:36:21.187826 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:21.187794    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:36:27.719059 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:27.719019    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:27.719518 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:27.719078    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:27.731875 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:27.731848    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:36:27.865833 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:36:27.865805    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:37:19.848267 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:19.848185    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:37:19.859753 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:19.859727    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:37:19.967983 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:19.967951    2564 scope.go:117] "RemoveContainer" containerID="2c10a83db4dc4a9855514ce40ffa86bd302ee89b52da15bb07ebf9cd2e8a3db5"
Jul 24 23:37:19.977171 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:19.977151    2564 scope.go:117] "RemoveContainer" containerID="df1f3da96808df496f0be8a5d7237628fa4fead6f0357e90f40fb2051ff1d068"
Jul 24 23:37:20.906165 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:20.906115    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:37:33.325324 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:33.325287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:42.797338 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:42.797295    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:37:46.457577 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:46.457537    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"]
Jul 24 23:37:46.479768 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:46.479735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:47.384108 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:47.384068    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"]
Jul 24 23:37:47.400575 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:47.400544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:48.357666 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.357617    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-140152715541424"]
Jul 24 23:37:48.361354 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.361334    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:48.363688 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.363661    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kzthv\""
Jul 24 23:37:48.367242 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.367221    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715541424"]
Jul 24 23:37:48.440573 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.440540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdlrl\" (UniqueName: \"kubernetes.io/projected/167749ef-45b3-46d8-8bdb-525a03269107-kube-api-access-sdlrl\") pod \"test-curl-7391-140152715541424\" (UID: \"167749ef-45b3-46d8-8bdb-525a03269107\") " pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:48.542091 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.542057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdlrl\" (UniqueName: \"kubernetes.io/projected/167749ef-45b3-46d8-8bdb-525a03269107-kube-api-access-sdlrl\") pod \"test-curl-7391-140152715541424\" (UID: \"167749ef-45b3-46d8-8bdb-525a03269107\") " pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:48.550686 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.550658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdlrl\" (UniqueName: \"kubernetes.io/projected/167749ef-45b3-46d8-8bdb-525a03269107-kube-api-access-sdlrl\") pod \"test-curl-7391-140152715541424\" (UID: \"167749ef-45b3-46d8-8bdb-525a03269107\") " pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:48.673650 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.673545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:48.817204 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.817179    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715541424"]
Jul 24 23:37:48.818966 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:37:48.818939    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod167749ef_45b3_46d8_8bdb_525a03269107.slice/crio-b06b64004ab56a3809d585e1b26ecccbbf6f041a76054cc9a01c2f1c143c4dcf WatchSource:0}: Error finding container b06b64004ab56a3809d585e1b26ecccbbf6f041a76054cc9a01c2f1c143c4dcf: Status 404 returned error can't find the container with id b06b64004ab56a3809d585e1b26ecccbbf6f041a76054cc9a01c2f1c143c4dcf
Jul 24 23:37:48.820285 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:48.820268    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:37:49.337611 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:49.337572    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715541424" event={"ID":"167749ef-45b3-46d8-8bdb-525a03269107","Type":"ContainerStarted","Data":"b06b64004ab56a3809d585e1b26ecccbbf6f041a76054cc9a01c2f1c143c4dcf"}
Jul 24 23:37:50.349489 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:50.349449    2564 generic.go:356] "Generic (PLEG): container finished" podID="167749ef-45b3-46d8-8bdb-525a03269107" containerID="79f38b25757629e4a4265ab3bd7b51d25087cae92b678b97b130160c8e5b4d3e" exitCode=0
Jul 24 23:37:50.349894 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:50.349529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715541424" event={"ID":"167749ef-45b3-46d8-8bdb-525a03269107","Type":"ContainerDied","Data":"79f38b25757629e4a4265ab3bd7b51d25087cae92b678b97b130160c8e5b4d3e"}
Jul 24 23:37:51.495931 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.495904    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:51.519265 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.519229    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-140152715541424_167749ef-45b3-46d8-8bdb-525a03269107/test-curl-7391-140152715541424/0.log"
Jul 24 23:37:51.550199 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.550165    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-140152715541424"]
Jul 24 23:37:51.553606 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.553576    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-140152715541424"]
Jul 24 23:37:51.574860 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.574834    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdlrl\" (UniqueName: \"kubernetes.io/projected/167749ef-45b3-46d8-8bdb-525a03269107-kube-api-access-sdlrl\") pod \"167749ef-45b3-46d8-8bdb-525a03269107\" (UID: \"167749ef-45b3-46d8-8bdb-525a03269107\") "
Jul 24 23:37:51.577007 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.576967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/167749ef-45b3-46d8-8bdb-525a03269107-kube-api-access-sdlrl" (OuterVolumeSpecName: "kube-api-access-sdlrl") pod "167749ef-45b3-46d8-8bdb-525a03269107" (UID: "167749ef-45b3-46d8-8bdb-525a03269107"). InnerVolumeSpecName "kube-api-access-sdlrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:37:51.676476 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.676386    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdlrl\" (UniqueName: \"kubernetes.io/projected/167749ef-45b3-46d8-8bdb-525a03269107-kube-api-access-sdlrl\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:37:51.687948 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.687906    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-140152715541552"]
Jul 24 23:37:51.688423 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.688409    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="167749ef-45b3-46d8-8bdb-525a03269107" containerName="test-curl-7391-140152715541424"
Jul 24 23:37:51.688481 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.688425    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="167749ef-45b3-46d8-8bdb-525a03269107" containerName="test-curl-7391-140152715541424"
Jul 24 23:37:51.688548 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.688539    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="167749ef-45b3-46d8-8bdb-525a03269107" containerName="test-curl-7391-140152715541424"
Jul 24 23:37:51.692227 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.692198    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:51.699838 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.699815    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="167749ef-45b3-46d8-8bdb-525a03269107" path="/var/lib/kubelet/pods/167749ef-45b3-46d8-8bdb-525a03269107/volumes"
Jul 24 23:37:51.709582 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.709559    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715541552"]
Jul 24 23:37:51.777652 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.777614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwlh6\" (UniqueName: \"kubernetes.io/projected/833a121a-9b6b-446c-b47b-4e9056979689-kube-api-access-vwlh6\") pod \"test-curl-7391-140152715541552\" (UID: \"833a121a-9b6b-446c-b47b-4e9056979689\") " pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:51.878365 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.878326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwlh6\" (UniqueName: \"kubernetes.io/projected/833a121a-9b6b-446c-b47b-4e9056979689-kube-api-access-vwlh6\") pod \"test-curl-7391-140152715541552\" (UID: \"833a121a-9b6b-446c-b47b-4e9056979689\") " pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:51.891675 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:51.891641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwlh6\" (UniqueName: \"kubernetes.io/projected/833a121a-9b6b-446c-b47b-4e9056979689-kube-api-access-vwlh6\") pod \"test-curl-7391-140152715541552\" (UID: \"833a121a-9b6b-446c-b47b-4e9056979689\") " pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:52.002412 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:52.002305    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:52.143399 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:52.143355    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715541552"]
Jul 24 23:37:52.151300 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:37:52.151204    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod833a121a_9b6b_446c_b47b_4e9056979689.slice/crio-9785885c20c30d6060e0ee9ce3f161a4ef982c9e8a306a3df8bff7a9b7a7109a WatchSource:0}: Error finding container 9785885c20c30d6060e0ee9ce3f161a4ef982c9e8a306a3df8bff7a9b7a7109a: Status 404 returned error can't find the container with id 9785885c20c30d6060e0ee9ce3f161a4ef982c9e8a306a3df8bff7a9b7a7109a
Jul 24 23:37:52.363005 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:52.362973    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541424"
Jul 24 23:37:52.363242 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:52.362973    2564 scope.go:117] "RemoveContainer" containerID="79f38b25757629e4a4265ab3bd7b51d25087cae92b678b97b130160c8e5b4d3e"
Jul 24 23:37:52.364458 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:52.364427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715541552" event={"ID":"833a121a-9b6b-446c-b47b-4e9056979689","Type":"ContainerStarted","Data":"9785885c20c30d6060e0ee9ce3f161a4ef982c9e8a306a3df8bff7a9b7a7109a"}
Jul 24 23:37:53.371583 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:53.371548    2564 generic.go:356] "Generic (PLEG): container finished" podID="833a121a-9b6b-446c-b47b-4e9056979689" containerID="c2591b5e411b809faac2a0dadcac858240712e96224569a3638424d64d464d58" exitCode=0
Jul 24 23:37:53.372098 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:53.371632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715541552" event={"ID":"833a121a-9b6b-446c-b47b-4e9056979689","Type":"ContainerDied","Data":"c2591b5e411b809faac2a0dadcac858240712e96224569a3638424d64d464d58"}
Jul 24 23:37:54.516636 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.516610    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:54.538754 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.538728    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-140152715541552_833a121a-9b6b-446c-b47b-4e9056979689/test-curl-7391-140152715541552/0.log"
Jul 24 23:37:54.566869 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.566837    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-140152715541552"]
Jul 24 23:37:54.570154 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.570112    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-140152715541552"]
Jul 24 23:37:54.600744 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.600701    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwlh6\" (UniqueName: \"kubernetes.io/projected/833a121a-9b6b-446c-b47b-4e9056979689-kube-api-access-vwlh6\") pod \"833a121a-9b6b-446c-b47b-4e9056979689\" (UID: \"833a121a-9b6b-446c-b47b-4e9056979689\") "
Jul 24 23:37:54.602906 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.602875    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/833a121a-9b6b-446c-b47b-4e9056979689-kube-api-access-vwlh6" (OuterVolumeSpecName: "kube-api-access-vwlh6") pod "833a121a-9b6b-446c-b47b-4e9056979689" (UID: "833a121a-9b6b-446c-b47b-4e9056979689"). InnerVolumeSpecName "kube-api-access-vwlh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:37:54.702115 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.702021    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwlh6\" (UniqueName: \"kubernetes.io/projected/833a121a-9b6b-446c-b47b-4e9056979689-kube-api-access-vwlh6\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:37:54.736004 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.735970    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:54.736429 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.736416    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833a121a-9b6b-446c-b47b-4e9056979689" containerName="test-curl-7391-140152715541552"
Jul 24 23:37:54.736488 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.736430    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="833a121a-9b6b-446c-b47b-4e9056979689" containerName="test-curl-7391-140152715541552"
Jul 24 23:37:54.736526 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.736514    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="833a121a-9b6b-446c-b47b-4e9056979689" containerName="test-curl-7391-140152715541552"
Jul 24 23:37:54.739636 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.739617    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:54.745887 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.745862    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:54.904758 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:54.904721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn6mx\" (UniqueName: \"kubernetes.io/projected/9ba2ef61-959f-4807-99a5-2e3e3eebea48-kube-api-access-mn6mx\") pod \"test-curl-7391-140152720381680\" (UID: \"9ba2ef61-959f-4807-99a5-2e3e3eebea48\") " pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:55.006186 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.006081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn6mx\" (UniqueName: \"kubernetes.io/projected/9ba2ef61-959f-4807-99a5-2e3e3eebea48-kube-api-access-mn6mx\") pod \"test-curl-7391-140152720381680\" (UID: \"9ba2ef61-959f-4807-99a5-2e3e3eebea48\") " pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:55.013810 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.013768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn6mx\" (UniqueName: \"kubernetes.io/projected/9ba2ef61-959f-4807-99a5-2e3e3eebea48-kube-api-access-mn6mx\") pod \"test-curl-7391-140152720381680\" (UID: \"9ba2ef61-959f-4807-99a5-2e3e3eebea48\") " pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:55.050653 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.050625    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:55.189010 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.188970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:55.191906 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:37:55.191882    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ba2ef61_959f_4807_99a5_2e3e3eebea48.slice/crio-3be6301dc979ae94d3c81228b3e8dae71b4b87d9d809f59a23a2af16ca1f8d6c WatchSource:0}: Error finding container 3be6301dc979ae94d3c81228b3e8dae71b4b87d9d809f59a23a2af16ca1f8d6c: Status 404 returned error can't find the container with id 3be6301dc979ae94d3c81228b3e8dae71b4b87d9d809f59a23a2af16ca1f8d6c
Jul 24 23:37:55.386047 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.386013    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:55.386047 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.386034    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9785885c20c30d6060e0ee9ce3f161a4ef982c9e8a306a3df8bff7a9b7a7109a"
Jul 24 23:37:55.387312 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.387287    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152720381680" event={"ID":"9ba2ef61-959f-4807-99a5-2e3e3eebea48","Type":"ContainerStarted","Data":"3be6301dc979ae94d3c81228b3e8dae71b4b87d9d809f59a23a2af16ca1f8d6c"}
Jul 24 23:37:55.396634 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:37:55.396610    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7391-140152715541552\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="833a121a-9b6b-446c-b47b-4e9056979689" pod="opendatahub/test-curl-7391-140152715541552"
Jul 24 23:37:55.696948 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:55.696867    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="833a121a-9b6b-446c-b47b-4e9056979689" path="/var/lib/kubelet/pods/833a121a-9b6b-446c-b47b-4e9056979689/volumes"
Jul 24 23:37:56.402995 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:56.402961    2564 generic.go:356] "Generic (PLEG): container finished" podID="9ba2ef61-959f-4807-99a5-2e3e3eebea48" containerID="6db1c9198f5f6a740a7a794f28c82c552a494ac89077775ec2373b8f8ad20516" exitCode=0
Jul 24 23:37:56.403189 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:56.403007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152720381680" event={"ID":"9ba2ef61-959f-4807-99a5-2e3e3eebea48","Type":"ContainerDied","Data":"6db1c9198f5f6a740a7a794f28c82c552a494ac89077775ec2373b8f8ad20516"}
Jul 24 23:37:57.558466 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.558439    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:57.577934 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.577909    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-140152720381680_9ba2ef61-959f-4807-99a5-2e3e3eebea48/test-curl-7391-140152720381680/0.log"
Jul 24 23:37:57.603087 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.603049    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:57.608970 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.608944    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:57.734096 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.734003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn6mx\" (UniqueName: \"kubernetes.io/projected/9ba2ef61-959f-4807-99a5-2e3e3eebea48-kube-api-access-mn6mx\") pod \"9ba2ef61-959f-4807-99a5-2e3e3eebea48\" (UID: \"9ba2ef61-959f-4807-99a5-2e3e3eebea48\") "
Jul 24 23:37:57.734517 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.734495    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:57.734922 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.734908    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ba2ef61-959f-4807-99a5-2e3e3eebea48" containerName="test-curl-7391-140152720381680"
Jul 24 23:37:57.734993 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.734923    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ba2ef61-959f-4807-99a5-2e3e3eebea48" containerName="test-curl-7391-140152720381680"
Jul 24 23:37:57.735044 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.735009    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ba2ef61-959f-4807-99a5-2e3e3eebea48" containerName="test-curl-7391-140152720381680"
Jul 24 23:37:57.736158 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.736133    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba2ef61-959f-4807-99a5-2e3e3eebea48-kube-api-access-mn6mx" (OuterVolumeSpecName: "kube-api-access-mn6mx") pod "9ba2ef61-959f-4807-99a5-2e3e3eebea48" (UID: "9ba2ef61-959f-4807-99a5-2e3e3eebea48"). InnerVolumeSpecName "kube-api-access-mn6mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:37:57.738214 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.738185    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:57.743260 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.743234    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:57.835591 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.835554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxx4m\" (UniqueName: \"kubernetes.io/projected/4afb202a-e605-44ee-bcdc-0a95a84a669b-kube-api-access-gxx4m\") pod \"test-curl-7391-140152720381680\" (UID: \"4afb202a-e605-44ee-bcdc-0a95a84a669b\") " pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:57.835747 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.835639    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn6mx\" (UniqueName: \"kubernetes.io/projected/9ba2ef61-959f-4807-99a5-2e3e3eebea48-kube-api-access-mn6mx\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:37:57.936434 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.936403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxx4m\" (UniqueName: \"kubernetes.io/projected/4afb202a-e605-44ee-bcdc-0a95a84a669b-kube-api-access-gxx4m\") pod \"test-curl-7391-140152720381680\" (UID: \"4afb202a-e605-44ee-bcdc-0a95a84a669b\") " pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:57.944582 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:57.944544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxx4m\" (UniqueName: \"kubernetes.io/projected/4afb202a-e605-44ee-bcdc-0a95a84a669b-kube-api-access-gxx4m\") pod \"test-curl-7391-140152720381680\" (UID: \"4afb202a-e605-44ee-bcdc-0a95a84a669b\") " pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:58.054594 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:58.054493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:58.194223 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:58.194190    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:37:58.195841 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:37:58.195815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4afb202a_e605_44ee_bcdc_0a95a84a669b.slice/crio-2d24c0b26a9f803602d6aa0c325f173b1df286d3ec42dd30b81521193ba8a365 WatchSource:0}: Error finding container 2d24c0b26a9f803602d6aa0c325f173b1df286d3ec42dd30b81521193ba8a365: Status 404 returned error can't find the container with id 2d24c0b26a9f803602d6aa0c325f173b1df286d3ec42dd30b81521193ba8a365
Jul 24 23:37:58.417713 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:58.417684    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:37:58.417937 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:58.417740    2564 scope.go:117] "RemoveContainer" containerID="6db1c9198f5f6a740a7a794f28c82c552a494ac89077775ec2373b8f8ad20516"
Jul 24 23:37:58.419056 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:58.418904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152720381680" event={"ID":"4afb202a-e605-44ee-bcdc-0a95a84a669b","Type":"ContainerStarted","Data":"2d24c0b26a9f803602d6aa0c325f173b1df286d3ec42dd30b81521193ba8a365"}
Jul 24 23:37:59.427873 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:59.427844    2564 generic.go:356] "Generic (PLEG): container finished" podID="4afb202a-e605-44ee-bcdc-0a95a84a669b" containerID="a2cb0648a8549c4f31618933fd13685028a511af9955d7582b27c28e5dfef770" exitCode=0
Jul 24 23:37:59.428272 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:59.427880    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152720381680" event={"ID":"4afb202a-e605-44ee-bcdc-0a95a84a669b","Type":"ContainerDied","Data":"a2cb0648a8549c4f31618933fd13685028a511af9955d7582b27c28e5dfef770"}
Jul 24 23:37:59.695250 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:37:59.695174    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba2ef61-959f-4807-99a5-2e3e3eebea48" path="/var/lib/kubelet/pods/9ba2ef61-959f-4807-99a5-2e3e3eebea48/volumes"
Jul 24 23:38:00.394460 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.394427    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:38:00.577742 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.577720    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:38:00.600271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.600243    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-140152720381680_4afb202a-e605-44ee-bcdc-0a95a84a669b/test-curl-7391-140152720381680/0.log"
Jul 24 23:38:00.628231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.628194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:38:00.631953 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.631927    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-140152720381680"]
Jul 24 23:38:00.752387 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.752302    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:00.752777 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.752762    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4afb202a-e605-44ee-bcdc-0a95a84a669b" containerName="test-curl-7391-140152720381680"
Jul 24 23:38:00.752777 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.752778    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4afb202a-e605-44ee-bcdc-0a95a84a669b" containerName="test-curl-7391-140152720381680"
Jul 24 23:38:00.752861 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.752839    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4afb202a-e605-44ee-bcdc-0a95a84a669b" containerName="test-curl-7391-140152720381680"
Jul 24 23:38:00.755842 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.755826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:00.760488 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.760443    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:00.763819 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.763793    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxx4m\" (UniqueName: \"kubernetes.io/projected/4afb202a-e605-44ee-bcdc-0a95a84a669b-kube-api-access-gxx4m\") pod \"4afb202a-e605-44ee-bcdc-0a95a84a669b\" (UID: \"4afb202a-e605-44ee-bcdc-0a95a84a669b\") "
Jul 24 23:38:00.765999 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.765967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afb202a-e605-44ee-bcdc-0a95a84a669b-kube-api-access-gxx4m" (OuterVolumeSpecName: "kube-api-access-gxx4m") pod "4afb202a-e605-44ee-bcdc-0a95a84a669b" (UID: "4afb202a-e605-44ee-bcdc-0a95a84a669b"). InnerVolumeSpecName "kube-api-access-gxx4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:38:00.865578 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.865542    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9vv\" (UniqueName: \"kubernetes.io/projected/7ba597c3-6b35-401f-8f25-b606dc9c4f57-kube-api-access-rx9vv\") pod \"test-curl-7391-140152715593168\" (UID: \"7ba597c3-6b35-401f-8f25-b606dc9c4f57\") " pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:00.865748 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.865723    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxx4m\" (UniqueName: \"kubernetes.io/projected/4afb202a-e605-44ee-bcdc-0a95a84a669b-kube-api-access-gxx4m\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:00.966885 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.966846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9vv\" (UniqueName: \"kubernetes.io/projected/7ba597c3-6b35-401f-8f25-b606dc9c4f57-kube-api-access-rx9vv\") pod \"test-curl-7391-140152715593168\" (UID: \"7ba597c3-6b35-401f-8f25-b606dc9c4f57\") " pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:00.974845 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:00.974817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9vv\" (UniqueName: \"kubernetes.io/projected/7ba597c3-6b35-401f-8f25-b606dc9c4f57-kube-api-access-rx9vv\") pod \"test-curl-7391-140152715593168\" (UID: \"7ba597c3-6b35-401f-8f25-b606dc9c4f57\") " pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:01.067256 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.067174    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:01.210054 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.210026    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:01.211215 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:38:01.211189    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ba597c3_6b35_401f_8f25_b606dc9c4f57.slice/crio-0c6bad7b5428c76551599effab33d1bc6a90a8e4329c5b84a3688027e40f4f38 WatchSource:0}: Error finding container 0c6bad7b5428c76551599effab33d1bc6a90a8e4329c5b84a3688027e40f4f38: Status 404 returned error can't find the container with id 0c6bad7b5428c76551599effab33d1bc6a90a8e4329c5b84a3688027e40f4f38
Jul 24 23:38:01.442467 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.442433    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:38:01.442685 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.442437    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d24c0b26a9f803602d6aa0c325f173b1df286d3ec42dd30b81521193ba8a365"
Jul 24 23:38:01.443772 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.443744    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715593168" event={"ID":"7ba597c3-6b35-401f-8f25-b606dc9c4f57","Type":"ContainerStarted","Data":"0c6bad7b5428c76551599effab33d1bc6a90a8e4329c5b84a3688027e40f4f38"}
Jul 24 23:38:01.453611 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:38:01.453587    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7391-140152720381680\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="4afb202a-e605-44ee-bcdc-0a95a84a669b" pod="opendatahub/test-curl-7391-140152720381680"
Jul 24 23:38:01.477472 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.477436    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm" podUID="fe30c4d8-8ef7-4777-af04-a7657990d05b" containerName="main" containerID="cri-o://7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285" gracePeriod=45
Jul 24 23:38:01.698217 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.698147    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4afb202a-e605-44ee-bcdc-0a95a84a669b" path="/var/lib/kubelet/pods/4afb202a-e605-44ee-bcdc-0a95a84a669b/volumes"
Jul 24 23:38:01.744671 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.744648    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:38:01.774370 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774340    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-dshm\") pod \"fe30c4d8-8ef7-4777-af04-a7657990d05b\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") "
Jul 24 23:38:01.774549 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774401    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-tmp-dir\") pod \"fe30c4d8-8ef7-4777-af04-a7657990d05b\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") "
Jul 24 23:38:01.774549 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774477    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe30c4d8-8ef7-4777-af04-a7657990d05b-tls-certs\") pod \"fe30c4d8-8ef7-4777-af04-a7657990d05b\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") "
Jul 24 23:38:01.774549 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774509    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzjpv\" (UniqueName: \"kubernetes.io/projected/fe30c4d8-8ef7-4777-af04-a7657990d05b-kube-api-access-vzjpv\") pod \"fe30c4d8-8ef7-4777-af04-a7657990d05b\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") "
Jul 24 23:38:01.774549 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774546    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-model-cache\") pod \"fe30c4d8-8ef7-4777-af04-a7657990d05b\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") "
Jul 24 23:38:01.774753 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774594    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-home\") pod \"fe30c4d8-8ef7-4777-af04-a7657990d05b\" (UID: \"fe30c4d8-8ef7-4777-af04-a7657990d05b\") "
Jul 24 23:38:01.774753 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774681    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe30c4d8-8ef7-4777-af04-a7657990d05b" (UID: "fe30c4d8-8ef7-4777-af04-a7657990d05b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:01.774882 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774841    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe30c4d8-8ef7-4777-af04-a7657990d05b" (UID: "fe30c4d8-8ef7-4777-af04-a7657990d05b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:01.774937 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774887    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-home" (OuterVolumeSpecName: "home") pod "fe30c4d8-8ef7-4777-af04-a7657990d05b" (UID: "fe30c4d8-8ef7-4777-af04-a7657990d05b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:01.774937 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.774922    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:01.776539 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.776518    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-dshm" (OuterVolumeSpecName: "dshm") pod "fe30c4d8-8ef7-4777-af04-a7657990d05b" (UID: "fe30c4d8-8ef7-4777-af04-a7657990d05b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:01.776661 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.776643    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe30c4d8-8ef7-4777-af04-a7657990d05b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe30c4d8-8ef7-4777-af04-a7657990d05b" (UID: "fe30c4d8-8ef7-4777-af04-a7657990d05b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:38:01.776980 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.776951    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe30c4d8-8ef7-4777-af04-a7657990d05b-kube-api-access-vzjpv" (OuterVolumeSpecName: "kube-api-access-vzjpv") pod "fe30c4d8-8ef7-4777-af04-a7657990d05b" (UID: "fe30c4d8-8ef7-4777-af04-a7657990d05b"). InnerVolumeSpecName "kube-api-access-vzjpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:38:01.876400 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.876367    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:01.876400 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.876393    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe30c4d8-8ef7-4777-af04-a7657990d05b-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:01.876400 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.876405    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzjpv\" (UniqueName: \"kubernetes.io/projected/fe30c4d8-8ef7-4777-af04-a7657990d05b-kube-api-access-vzjpv\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:01.876632 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.876414    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:01.876632 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:01.876423    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe30c4d8-8ef7-4777-af04-a7657990d05b-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:02.403698 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.403650    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd" podUID="08a8f1e7-d7d6-443a-8c08-1947af9d38ae" containerName="main" containerID="cri-o://178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4" gracePeriod=45
Jul 24 23:38:02.452033 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.452000    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe30c4d8-8ef7-4777-af04-a7657990d05b" containerID="7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285" exitCode=0
Jul 24 23:38:02.452257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.452044    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"
Jul 24 23:38:02.452257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.452086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm" event={"ID":"fe30c4d8-8ef7-4777-af04-a7657990d05b","Type":"ContainerDied","Data":"7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285"}
Jul 24 23:38:02.452257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.452160    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm" event={"ID":"fe30c4d8-8ef7-4777-af04-a7657990d05b","Type":"ContainerDied","Data":"49e08d60fe5376439c28229dcdfb584215e5d2b37dacf2f493fa4d637b37e2f2"}
Jul 24 23:38:02.452257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.452188    2564 scope.go:117] "RemoveContainer" containerID="7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285"
Jul 24 23:38:02.453838 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.453811    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ba597c3-6b35-401f-8f25-b606dc9c4f57" containerID="b632cd22dbbcd03dedae62c427d8e4b928e7db0fb204d869a5eb749e55f1b126" exitCode=0
Jul 24 23:38:02.453925 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.453852    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715593168" event={"ID":"7ba597c3-6b35-401f-8f25-b606dc9c4f57","Type":"ContainerDied","Data":"b632cd22dbbcd03dedae62c427d8e4b928e7db0fb204d869a5eb749e55f1b126"}
Jul 24 23:38:02.484434 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.484400    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"]
Jul 24 23:38:02.488287 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.488261    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fe475f/rate-test-model-fe475f-kserve-788df45c9f-6n8hm"]
Jul 24 23:38:02.515061 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.514926    2564 scope.go:117] "RemoveContainer" containerID="7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285"
Jul 24 23:38:02.515302 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:38:02.515282    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285\": container with ID starting with 7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285 not found: ID does not exist" containerID="7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285"
Jul 24 23:38:02.515367 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.515322    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285"} err="failed to get container status \"7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285\": rpc error: code = NotFound desc = could not find container \"7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285\": container with ID starting with 7e070ee65139763637087f646828e186b417729d74d577eab357adff2bfe7285 not found: ID does not exist"
Jul 24 23:38:02.675636 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.675614    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:38:02.684673 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684650    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-model-cache\") pod \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") "
Jul 24 23:38:02.684791 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684679    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tmp-dir\") pod \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") "
Jul 24 23:38:02.684791 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684701    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tls-certs\") pod \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") "
Jul 24 23:38:02.684791 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684726    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-dshm\") pod \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") "
Jul 24 23:38:02.684791 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684746    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-home\") pod \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") "
Jul 24 23:38:02.684998 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684814    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5th8\" (UniqueName: \"kubernetes.io/projected/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-kube-api-access-m5th8\") pod \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\" (UID: \"08a8f1e7-d7d6-443a-8c08-1947af9d38ae\") "
Jul 24 23:38:02.684998 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684887    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "08a8f1e7-d7d6-443a-8c08-1947af9d38ae" (UID: "08a8f1e7-d7d6-443a-8c08-1947af9d38ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:02.684998 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.684932    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "08a8f1e7-d7d6-443a-8c08-1947af9d38ae" (UID: "08a8f1e7-d7d6-443a-8c08-1947af9d38ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:02.685177 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.685057    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-home" (OuterVolumeSpecName: "home") pod "08a8f1e7-d7d6-443a-8c08-1947af9d38ae" (UID: "08a8f1e7-d7d6-443a-8c08-1947af9d38ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:02.685239 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.685220    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:02.685286 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.685239    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:02.685286 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.685253    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:02.686916 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.686893    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-dshm" (OuterVolumeSpecName: "dshm") pod "08a8f1e7-d7d6-443a-8c08-1947af9d38ae" (UID: "08a8f1e7-d7d6-443a-8c08-1947af9d38ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:38:02.687008 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.686891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "08a8f1e7-d7d6-443a-8c08-1947af9d38ae" (UID: "08a8f1e7-d7d6-443a-8c08-1947af9d38ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:38:02.687008 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.686936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-kube-api-access-m5th8" (OuterVolumeSpecName: "kube-api-access-m5th8") pod "08a8f1e7-d7d6-443a-8c08-1947af9d38ae" (UID: "08a8f1e7-d7d6-443a-8c08-1947af9d38ae"). InnerVolumeSpecName "kube-api-access-m5th8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:38:02.786484 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.786444    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:02.786484 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.786482    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:02.786484 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:02.786492    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5th8\" (UniqueName: \"kubernetes.io/projected/08a8f1e7-d7d6-443a-8c08-1947af9d38ae-kube-api-access-m5th8\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:03.461444 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.461407    2564 generic.go:356] "Generic (PLEG): container finished" podID="08a8f1e7-d7d6-443a-8c08-1947af9d38ae" containerID="178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4" exitCode=0
Jul 24 23:38:03.461604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.461450    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"
Jul 24 23:38:03.461604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.461496    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd" event={"ID":"08a8f1e7-d7d6-443a-8c08-1947af9d38ae","Type":"ContainerDied","Data":"178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4"}
Jul 24 23:38:03.461604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.461539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd" event={"ID":"08a8f1e7-d7d6-443a-8c08-1947af9d38ae","Type":"ContainerDied","Data":"c57082e7ca3a9d699de934d27e98053fb879fb906002052b7f340facfd05d6ea"}
Jul 24 23:38:03.461604 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.461562    2564 scope.go:117] "RemoveContainer" containerID="178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4"
Jul 24 23:38:03.472449 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.472429    2564 scope.go:117] "RemoveContainer" containerID="178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4"
Jul 24 23:38:03.472727 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:38:03.472708    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4\": container with ID starting with 178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4 not found: ID does not exist" containerID="178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4"
Jul 24 23:38:03.472801 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.472737    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4"} err="failed to get container status \"178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4\": rpc error: code = NotFound desc = could not find container \"178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4\": container with ID starting with 178abe4a67ee6bf28224c76ff4a7ea13add0b38db42f2e47453f1e737b5e4ee4 not found: ID does not exist"
Jul 24 23:38:03.486382 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.486352    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"]
Jul 24 23:38:03.490539 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.490515    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6f3ce2/rate-test-model-6f3ce2-kserve-77677476b7-8d6jd"]
Jul 24 23:38:03.614828 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.614803    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:03.636726 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.636697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-140152715593168_7ba597c3-6b35-401f-8f25-b606dc9c4f57/test-curl-7391-140152715593168/0.log"
Jul 24 23:38:03.662518 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.662488    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:03.666167 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.666140    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:03.700486 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.695612    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx9vv\" (UniqueName: \"kubernetes.io/projected/7ba597c3-6b35-401f-8f25-b606dc9c4f57-kube-api-access-rx9vv\") pod \"7ba597c3-6b35-401f-8f25-b606dc9c4f57\" (UID: \"7ba597c3-6b35-401f-8f25-b606dc9c4f57\") "
Jul 24 23:38:03.701366 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.701334    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba597c3-6b35-401f-8f25-b606dc9c4f57-kube-api-access-rx9vv" (OuterVolumeSpecName: "kube-api-access-rx9vv") pod "7ba597c3-6b35-401f-8f25-b606dc9c4f57" (UID: "7ba597c3-6b35-401f-8f25-b606dc9c4f57"). InnerVolumeSpecName "kube-api-access-rx9vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:38:03.703391 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.703358    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08a8f1e7-d7d6-443a-8c08-1947af9d38ae" path="/var/lib/kubelet/pods/08a8f1e7-d7d6-443a-8c08-1947af9d38ae/volumes"
Jul 24 23:38:03.703773 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.703754    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ba597c3-6b35-401f-8f25-b606dc9c4f57" path="/var/lib/kubelet/pods/7ba597c3-6b35-401f-8f25-b606dc9c4f57/volumes"
Jul 24 23:38:03.704226 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.704206    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe30c4d8-8ef7-4777-af04-a7657990d05b" path="/var/lib/kubelet/pods/fe30c4d8-8ef7-4777-af04-a7657990d05b/volumes"
Jul 24 23:38:03.790627 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.790593    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:03.791015 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791002    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a8f1e7-d7d6-443a-8c08-1947af9d38ae" containerName="main"
Jul 24 23:38:03.791015 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791016    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a8f1e7-d7d6-443a-8c08-1947af9d38ae" containerName="main"
Jul 24 23:38:03.791088 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791032    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba597c3-6b35-401f-8f25-b606dc9c4f57" containerName="test-curl-7391-140152715593168"
Jul 24 23:38:03.791088 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791039    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba597c3-6b35-401f-8f25-b606dc9c4f57" containerName="test-curl-7391-140152715593168"
Jul 24 23:38:03.791088 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791048    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe30c4d8-8ef7-4777-af04-a7657990d05b" containerName="main"
Jul 24 23:38:03.791088 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791054    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe30c4d8-8ef7-4777-af04-a7657990d05b" containerName="main"
Jul 24 23:38:03.791231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791116    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe30c4d8-8ef7-4777-af04-a7657990d05b" containerName="main"
Jul 24 23:38:03.791231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791148    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba597c3-6b35-401f-8f25-b606dc9c4f57" containerName="test-curl-7391-140152715593168"
Jul 24 23:38:03.791231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.791155    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a8f1e7-d7d6-443a-8c08-1947af9d38ae" containerName="main"
Jul 24 23:38:03.795388 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.795371    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:03.797102 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.797074    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx9vv\" (UniqueName: \"kubernetes.io/projected/7ba597c3-6b35-401f-8f25-b606dc9c4f57-kube-api-access-rx9vv\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:03.800440 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.800415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:03.897966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.897930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z5bh\" (UniqueName: \"kubernetes.io/projected/1104cb7e-60ec-49a4-acce-5a938f3ad1e9-kube-api-access-5z5bh\") pod \"test-curl-7391-140152715593168\" (UID: \"1104cb7e-60ec-49a4-acce-5a938f3ad1e9\") " pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:03.998960 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:03.998869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z5bh\" (UniqueName: \"kubernetes.io/projected/1104cb7e-60ec-49a4-acce-5a938f3ad1e9-kube-api-access-5z5bh\") pod \"test-curl-7391-140152715593168\" (UID: \"1104cb7e-60ec-49a4-acce-5a938f3ad1e9\") " pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:04.006726 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:04.006703    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z5bh\" (UniqueName: \"kubernetes.io/projected/1104cb7e-60ec-49a4-acce-5a938f3ad1e9-kube-api-access-5z5bh\") pod \"test-curl-7391-140152715593168\" (UID: \"1104cb7e-60ec-49a4-acce-5a938f3ad1e9\") " pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:04.105846 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:04.105813    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:04.248040 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:04.248014    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:04.250180 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:38:04.250097    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1104cb7e_60ec_49a4_acce_5a938f3ad1e9.slice/crio-622817d973e3e87ca9cc9d5b1c528c79cc2784f8801ce09383f48076a5161266 WatchSource:0}: Error finding container 622817d973e3e87ca9cc9d5b1c528c79cc2784f8801ce09383f48076a5161266: Status 404 returned error can't find the container with id 622817d973e3e87ca9cc9d5b1c528c79cc2784f8801ce09383f48076a5161266
Jul 24 23:38:04.469979 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:04.469946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715593168" event={"ID":"1104cb7e-60ec-49a4-acce-5a938f3ad1e9","Type":"ContainerStarted","Data":"622817d973e3e87ca9cc9d5b1c528c79cc2784f8801ce09383f48076a5161266"}
Jul 24 23:38:04.471315 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:04.471295    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:04.471315 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:04.471304    2564 scope.go:117] "RemoveContainer" containerID="b632cd22dbbcd03dedae62c427d8e4b928e7db0fb204d869a5eb749e55f1b126"
Jul 24 23:38:05.480935 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:05.480899    2564 generic.go:356] "Generic (PLEG): container finished" podID="1104cb7e-60ec-49a4-acce-5a938f3ad1e9" containerID="9cad1cde03edde38b967356a1f12db5bd20c5d2a02a443ea23d52b45be39c5ee" exitCode=0
Jul 24 23:38:05.481343 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:05.480979    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7391-140152715593168" event={"ID":"1104cb7e-60ec-49a4-acce-5a938f3ad1e9","Type":"ContainerDied","Data":"9cad1cde03edde38b967356a1f12db5bd20c5d2a02a443ea23d52b45be39c5ee"}
Jul 24 23:38:06.630461 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.630435    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:06.650625 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.650597    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7391-140152715593168_1104cb7e-60ec-49a4-acce-5a938f3ad1e9/test-curl-7391-140152715593168/0.log"
Jul 24 23:38:06.677339 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.677305    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:06.680888 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.680860    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7391-140152715593168"]
Jul 24 23:38:06.722844 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.722817    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z5bh\" (UniqueName: \"kubernetes.io/projected/1104cb7e-60ec-49a4-acce-5a938f3ad1e9-kube-api-access-5z5bh\") pod \"1104cb7e-60ec-49a4-acce-5a938f3ad1e9\" (UID: \"1104cb7e-60ec-49a4-acce-5a938f3ad1e9\") "
Jul 24 23:38:06.724928 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.724904    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1104cb7e-60ec-49a4-acce-5a938f3ad1e9-kube-api-access-5z5bh" (OuterVolumeSpecName: "kube-api-access-5z5bh") pod "1104cb7e-60ec-49a4-acce-5a938f3ad1e9" (UID: "1104cb7e-60ec-49a4-acce-5a938f3ad1e9"). InnerVolumeSpecName "kube-api-access-5z5bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:38:06.823699 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:06.823615    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z5bh\" (UniqueName: \"kubernetes.io/projected/1104cb7e-60ec-49a4-acce-5a938f3ad1e9-kube-api-access-5z5bh\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:38:07.195417 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.195383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:07.470515 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.470433    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"]
Jul 24 23:38:07.470932 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.470916    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1104cb7e-60ec-49a4-acce-5a938f3ad1e9" containerName="test-curl-7391-140152715593168"
Jul 24 23:38:07.470972 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.470935    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1104cb7e-60ec-49a4-acce-5a938f3ad1e9" containerName="test-curl-7391-140152715593168"
Jul 24 23:38:07.471024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.471015    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1104cb7e-60ec-49a4-acce-5a938f3ad1e9" containerName="test-curl-7391-140152715593168"
Jul 24 23:38:07.474185 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.474168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.476292 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.476274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-0aa0e6-openshift-default-dockercfg-wpq6d\""
Jul 24 23:38:07.485412 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.485388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"]
Jul 24 23:38:07.494210 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.494184    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:07.495885 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.495861    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="622817d973e3e87ca9cc9d5b1c528c79cc2784f8801ce09383f48076a5161266"
Jul 24 23:38:07.495995 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.495885    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:07.526949 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:38:07.525873    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7391-140152715593168\" is forbidden: User \"system:node:ip-10-0-143-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-78.ec2.internal' and this object" podUID="1104cb7e-60ec-49a4-acce-5a938f3ad1e9" pod="opendatahub/test-curl-7391-140152715593168"
Jul 24 23:38:07.529424 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-credential-socket\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529524 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529455    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-data\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529588 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-socket\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529588 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-token\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529658 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-certs\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529658 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istiod-ca-cert\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529658 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529623    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-envoy\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529771 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-podinfo\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.529771 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.529727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znqx4\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-kube-api-access-znqx4\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630469 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-certs\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630469 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istiod-ca-cert\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630503    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-envoy\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-podinfo\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znqx4\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-kube-api-access-znqx4\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-credential-socket\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-data\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-socket\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-token\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.630966 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-certs\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.631375 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.630983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-socket\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.631375 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.631059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-data\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.631375 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.631244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-credential-socket\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.631685 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.631665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istiod-ca-cert\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.633001 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.632969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-envoy\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.633223 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.633203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-podinfo\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.638867 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.638837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-token\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.638964 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.638932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znqx4\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-kube-api-access-znqx4\") pod \"e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") " pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.696364 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.696329    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1104cb7e-60ec-49a4-acce-5a938f3ad1e9" path="/var/lib/kubelet/pods/1104cb7e-60ec-49a4-acce-5a938f3ad1e9/volumes"
Jul 24 23:38:07.725797 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.725701    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:07.786034 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.786004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:07.944761 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:38:07.944720    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bdf660_ff3b_4609_a83d_85d9e83e3e5e.slice/crio-3099939d96adc49fccb7136199b0c93d8f7758d9c39bbe3fdbb12a53ecb3f032 WatchSource:0}: Error finding container 3099939d96adc49fccb7136199b0c93d8f7758d9c39bbe3fdbb12a53ecb3f032: Status 404 returned error can't find the container with id 3099939d96adc49fccb7136199b0c93d8f7758d9c39bbe3fdbb12a53ecb3f032
Jul 24 23:38:07.945493 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.944806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"]
Jul 24 23:38:07.947642 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.947606    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:07.947710 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.947691    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:07.947760 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:07.947736    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:08.503978 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:08.503942    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" event={"ID":"37bdf660-ff3b-4609-a83d-85d9e83e3e5e","Type":"ContainerStarted","Data":"82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"}
Jul 24 23:38:08.504193 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:08.503995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" event={"ID":"37bdf660-ff3b-4609-a83d-85d9e83e3e5e","Type":"ContainerStarted","Data":"3099939d96adc49fccb7136199b0c93d8f7758d9c39bbe3fdbb12a53ecb3f032"}
Jul 24 23:38:08.524064 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:08.524011    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podStartSLOduration=1.523994829 podStartE2EDuration="1.523994829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:08.523203016 +0000 UTC m=+2749.376529556" watchObservedRunningTime="2026-07-24 23:38:08.523994829 +0000 UTC m=+2749.377321427"
Jul 24 23:38:08.786853 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:08.786767    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:09.792003 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:09.791960    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:10.463688 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:10.463655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:10.516426 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:10.516400    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:10.517582 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:10.517560    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:10.545602 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:10.545564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:10.716984 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:10.716880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:11.095614 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:11.095589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:11.339535 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:11.339507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:11.436989 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:11.436880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:14.953846 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:14.953805    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:18.090029 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:18.089996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:38:25.856007 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:25.855973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:26.163873 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.163052    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": read tcp 10.132.0.2:39468->10.132.0.78:15021: read: connection reset by peer" start-of-body=
Jul 24 23:38:26.163873 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.163115    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": read tcp 10.132.0.2:39468->10.132.0.78:15021: read: connection reset by peer"
Jul 24 23:38:26.168293 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.168268    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:26.599701 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.599669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:26.622606 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.622573    2564 generic.go:356] "Generic (PLEG): container finished" podID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerID="82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de" exitCode=0
Jul 24 23:38:26.622806 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.622655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" event={"ID":"37bdf660-ff3b-4609-a83d-85d9e83e3e5e","Type":"ContainerDied","Data":"82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"}
Jul 24 23:38:26.623045 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.623031    2564 scope.go:117] "RemoveContainer" containerID="82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"
Jul 24 23:38:26.624078 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.624054    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:26.624173 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.624114    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:26.624173 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:26.624155    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:27.630273 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:27.630237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" event={"ID":"37bdf660-ff3b-4609-a83d-85d9e83e3e5e","Type":"ContainerStarted","Data":"824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7"}
Jul 24 23:38:27.786196 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:27.786158    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:27.786443 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:27.786421    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 24 23:38:27.786510 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:27.786492    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 24 23:38:28.787205 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:28.787168    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 24 23:38:28.787631 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:28.787227    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 24 23:38:29.171298 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.171265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:29.273407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.273353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:29.373605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.373568    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"]
Jul 24 23:38:29.378008 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.377982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.380774 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.380744    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-3fb847-serving-cert\""
Jul 24 23:38:29.380774 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.380764    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 23:38:29.381497 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.381484    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:38:29.381562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.381506    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49q74\""
Jul 24 23:38:29.385827 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.385802    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"]
Jul 24 23:38:29.406435 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.406403    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:29.457776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.457646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk5v7\" (UniqueName: \"kubernetes.io/projected/2f19712a-5771-45a2-aee2-ffb901f0a00e-kube-api-access-dk5v7\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.457776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.457754    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f19712a-5771-45a2-aee2-ffb901f0a00e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.458024 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.457804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f19712a-5771-45a2-aee2-ffb901f0a00e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.558758 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.558703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f19712a-5771-45a2-aee2-ffb901f0a00e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.559513 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.559472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk5v7\" (UniqueName: \"kubernetes.io/projected/2f19712a-5771-45a2-aee2-ffb901f0a00e-kube-api-access-dk5v7\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.559640 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.559527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f19712a-5771-45a2-aee2-ffb901f0a00e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.560309 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.560279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f19712a-5771-45a2-aee2-ffb901f0a00e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.562361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.562341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f19712a-5771-45a2-aee2-ffb901f0a00e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.574373 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.574351    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk5v7\" (UniqueName: \"kubernetes.io/projected/2f19712a-5771-45a2-aee2-ffb901f0a00e-kube-api-access-dk5v7\") pod \"maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.637567 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.637466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:29.691164 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.691116    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:29.786722 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.786687    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 24 23:38:29.786889 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.786761    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 24 23:38:29.903117 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.902964    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"]
Jul 24 23:38:29.905580 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:38:29.905550    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f19712a_5771_45a2_aee2_ffb901f0a00e.slice/crio-26a49eb91064f36d69fef70063fdc52676fbff64e629f047fa579ebe49125ec6 WatchSource:0}: Error finding container 26a49eb91064f36d69fef70063fdc52676fbff64e629f047fa579ebe49125ec6: Status 404 returned error can't find the container with id 26a49eb91064f36d69fef70063fdc52676fbff64e629f047fa579ebe49125ec6
Jul 24 23:38:29.947150 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:29.942665    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:30.058212 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:30.047299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:30.666657 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:30.666618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2" event={"ID":"2f19712a-5771-45a2-aee2-ffb901f0a00e","Type":"ContainerStarted","Data":"26a49eb91064f36d69fef70063fdc52676fbff64e629f047fa579ebe49125ec6"}
Jul 24 23:38:30.786437 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:30.786398    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 24 23:38:30.786639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:30.786468    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 24 23:38:31.790808 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:31.790775    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:31.791299 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:31.791162    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:31.792095 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:31.792074    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:38:32.681037 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:32.681000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2" event={"ID":"2f19712a-5771-45a2-aee2-ffb901f0a00e","Type":"ContainerStarted","Data":"2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d"}
Jul 24 23:38:32.681387 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:32.681362    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:32.700170 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:32.700109    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2" podStartSLOduration=1.102600175 podStartE2EDuration="3.700094158s" totalImagesPullingTime="2.597493983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:32.697843666 +0000 UTC m=+2773.551170206" watchObservedRunningTime="2026-07-24 23:38:32.700094158 +0000 UTC m=+2773.553420697"
Jul 24 23:38:33.613710 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:33.613662    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:38.693260 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:38.693215    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:38:39.397135 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:38:39.397095    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:39:07.697570 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.697535    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"]
Jul 24 23:39:07.703575 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.703552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.706096 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.706075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0aa0e6\"/\"kube-root-ca.crt\""
Jul 24 23:39:07.707018 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.706996    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0aa0e6\"/\"ipp-route-0aa0e6-kserve-self-signed-certs\""
Jul 24 23:39:07.707151 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.706996    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0aa0e6\"/\"default-dockercfg-xpjc5\""
Jul 24 23:39:07.707151 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.707039    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0aa0e6\"/\"openshift-service-ca.crt\""
Jul 24 23:39:07.708249 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.708224    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"]
Jul 24 23:39:07.723928 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.723902    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:07.835992 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.835961    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-dshm\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.836176 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.836026    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-home\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.836176 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.836055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txcxm\" (UniqueName: \"kubernetes.io/projected/80a2f455-0443-4745-9458-cca77b5560f5-kube-api-access-txcxm\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.836176 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.836106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-model-cache\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.836296 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.836204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-tmp-dir\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.836296 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.836233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2f455-0443-4745-9458-cca77b5560f5-tls-certs\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.937708 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.937668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-tmp-dir\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.937708 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.937708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2f455-0443-4745-9458-cca77b5560f5-tls-certs\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.937919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.937759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-dshm\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.937919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.937819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-home\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.937919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.937843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txcxm\" (UniqueName: \"kubernetes.io/projected/80a2f455-0443-4745-9458-cca77b5560f5-kube-api-access-txcxm\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.937919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.937901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-model-cache\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.938181 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.938161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-tmp-dir\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.938254 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.938206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-home\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.938610 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.938589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-model-cache\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.940443 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.940422    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-dshm\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.940672 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.940650    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2f455-0443-4745-9458-cca77b5560f5-tls-certs\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:07.946295 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:07.946270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txcxm\" (UniqueName: \"kubernetes.io/projected/80a2f455-0443-4745-9458-cca77b5560f5-kube-api-access-txcxm\") pod \"ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") " pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:08.014900 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:08.014811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:08.164667 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:08.164638    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"]
Jul 24 23:39:08.166302 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:39:08.166269    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80a2f455_0443_4745_9458_cca77b5560f5.slice/crio-ee0bd50fa60da596f821e0cfd758380a83339d323fb68179e0e19f576890df6c WatchSource:0}: Error finding container ee0bd50fa60da596f821e0cfd758380a83339d323fb68179e0e19f576890df6c: Status 404 returned error can't find the container with id ee0bd50fa60da596f821e0cfd758380a83339d323fb68179e0e19f576890df6c
Jul 24 23:39:08.903738 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:08.903707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b" event={"ID":"80a2f455-0443-4745-9458-cca77b5560f5","Type":"ContainerStarted","Data":"aa42dbaf4833d0ad1a8fa578e174afce55ff0ccd777a1c33b2855dcb7a9c4042"}
Jul 24 23:39:08.903738 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:08.903748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b" event={"ID":"80a2f455-0443-4745-9458-cca77b5560f5","Type":"ContainerStarted","Data":"ee0bd50fa60da596f821e0cfd758380a83339d323fb68179e0e19f576890df6c"}
Jul 24 23:39:08.920589 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:08.920539    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b" podStartSLOduration=1.920520975 podStartE2EDuration="1.920520975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:39:08.919369919 +0000 UTC m=+2809.772696482" watchObservedRunningTime="2026-07-24 23:39:08.920520975 +0000 UTC m=+2809.773847508"
Jul 24 23:39:11.900649 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:11.900604    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:39:17.571562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:17.571508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:18.015186 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:18.015151    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:18.015413 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:18.015284    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:18.027718 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:18.027687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:18.979469 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:18.979437    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:39:27.189866 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:27.189832    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:39:38.264867 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:38.264819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:38.586840 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:38.586809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:38.978815 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:38.978721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:41.463698 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:41.463665    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:41.548257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:41.548208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:41.618335 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:41.618298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:41.908644 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:41.908606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:42.141615 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:42.141580    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:42.230674 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:42.230595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:45.968209 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:45.968170    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:47.202642 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:47.202600    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:39:56.894733 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:56.894696    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"]
Jul 24 23:39:56.898605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:56.898585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:39:56.901450 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:56.901421    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:39:56.908056 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:56.908026    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"]
Jul 24 23:39:56.919905 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:56.919877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42317e21-131d-40fb-bd35-4b775bb05afb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8\" (UID: \"42317e21-131d-40fb-bd35-4b775bb05afb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:39:57.020555 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:57.020522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42317e21-131d-40fb-bd35-4b775bb05afb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8\" (UID: \"42317e21-131d-40fb-bd35-4b775bb05afb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:39:57.021179 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:57.021159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42317e21-131d-40fb-bd35-4b775bb05afb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8\" (UID: \"42317e21-131d-40fb-bd35-4b775bb05afb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:39:57.210615 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:57.210533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:39:57.348824 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:57.348800    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"]
Jul 24 23:39:57.350915 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:39:57.350887    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42317e21_131d_40fb_bd35_4b775bb05afb.slice/crio-48b9ced5749e86fbc49e8add7b8de48932905efadb62613d4ab80790c652de62 WatchSource:0}: Error finding container 48b9ced5749e86fbc49e8add7b8de48932905efadb62613d4ab80790c652de62: Status 404 returned error can't find the container with id 48b9ced5749e86fbc49e8add7b8de48932905efadb62613d4ab80790c652de62
Jul 24 23:39:58.225522 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:58.225488    2564 generic.go:356] "Generic (PLEG): container finished" podID="42317e21-131d-40fb-bd35-4b775bb05afb" containerID="ce437366dd7208459c8385b88d2f05738acaec2ebc131ca06249b9a9c1b9ea0f" exitCode=0
Jul 24 23:39:58.225933 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:58.225577    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8" event={"ID":"42317e21-131d-40fb-bd35-4b775bb05afb","Type":"ContainerDied","Data":"ce437366dd7208459c8385b88d2f05738acaec2ebc131ca06249b9a9c1b9ea0f"}
Jul 24 23:39:58.225933 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:58.225644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8" event={"ID":"42317e21-131d-40fb-bd35-4b775bb05afb","Type":"ContainerStarted","Data":"48b9ced5749e86fbc49e8add7b8de48932905efadb62613d4ab80790c652de62"}
Jul 24 23:39:59.377249 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:59.377225    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:39:59.439366 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:59.439328    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42317e21-131d-40fb-bd35-4b775bb05afb-maas-api-service-ca\") pod \"42317e21-131d-40fb-bd35-4b775bb05afb\" (UID: \"42317e21-131d-40fb-bd35-4b775bb05afb\") "
Jul 24 23:39:59.439700 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:59.439679    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42317e21-131d-40fb-bd35-4b775bb05afb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42317e21-131d-40fb-bd35-4b775bb05afb" (UID: "42317e21-131d-40fb-bd35-4b775bb05afb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:39:59.540928 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:39:59.540845    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42317e21-131d-40fb-bd35-4b775bb05afb-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:40:00.240186 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:00.240154    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"
Jul 24 23:40:00.240186 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:00.240178    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8" event={"ID":"42317e21-131d-40fb-bd35-4b775bb05afb","Type":"ContainerDied","Data":"48b9ced5749e86fbc49e8add7b8de48932905efadb62613d4ab80790c652de62"}
Jul 24 23:40:00.240459 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:00.240228    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48b9ced5749e86fbc49e8add7b8de48932905efadb62613d4ab80790c652de62"
Jul 24 23:40:07.435562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:07.435526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:07.505308 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:07.505274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:07.606270 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:07.606235    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:11.970078 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:11.969989    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"]
Jul 24 23:40:11.972677 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:11.972655    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4bb47e-cmzl8"]
Jul 24 23:40:11.973965 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:11.973943    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:12.091275 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:12.091237    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:40:13.286751 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:13.286714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:13.696410 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:13.696370    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42317e21-131d-40fb-bd35-4b775bb05afb" path="/var/lib/kubelet/pods/42317e21-131d-40fb-bd35-4b775bb05afb/volumes"
Jul 24 23:40:18.008891 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:18.008860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:18.110459 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:18.110423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:18.274618 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:18.274542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:18.494858 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:18.494820    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:21.978453 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.978418    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"]
Jul 24 23:40:21.978919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.978873    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42317e21-131d-40fb-bd35-4b775bb05afb" containerName="revoke-keys"
Jul 24 23:40:21.978919 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.978885    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="42317e21-131d-40fb-bd35-4b775bb05afb" containerName="revoke-keys"
Jul 24 23:40:21.978992 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.978959    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="42317e21-131d-40fb-bd35-4b775bb05afb" containerName="revoke-keys"
Jul 24 23:40:21.982362 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.982344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:21.984741 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.984716    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:40:21.988835 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:21.988811    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"]
Jul 24 23:40:22.069749 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:22.069684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15000f24-3a77-44b2-aa10-6e34b1608c69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m\" (UID: \"15000f24-3a77-44b2-aa10-6e34b1608c69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:22.171113 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:22.171076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15000f24-3a77-44b2-aa10-6e34b1608c69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m\" (UID: \"15000f24-3a77-44b2-aa10-6e34b1608c69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:22.171726 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:22.171705    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15000f24-3a77-44b2-aa10-6e34b1608c69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m\" (UID: \"15000f24-3a77-44b2-aa10-6e34b1608c69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:22.295422 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:22.295327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:22.442450 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:22.442413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"]
Jul 24 23:40:22.443031 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:40:22.442992    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15000f24_3a77_44b2_aa10_6e34b1608c69.slice/crio-167e68449f58f786ee64f20144343ea93d85ee2751a82239980a4ae4bb9b41fd WatchSource:0}: Error finding container 167e68449f58f786ee64f20144343ea93d85ee2751a82239980a4ae4bb9b41fd: Status 404 returned error can't find the container with id 167e68449f58f786ee64f20144343ea93d85ee2751a82239980a4ae4bb9b41fd
Jul 24 23:40:23.385518 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:23.385485    2564 generic.go:356] "Generic (PLEG): container finished" podID="15000f24-3a77-44b2-aa10-6e34b1608c69" containerID="591fbc2cd30b70027bfcfcba1f470dc3a44f1b18b1328e6c4ec274b4ed22c576" exitCode=0
Jul 24 23:40:23.385898 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:23.385525    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m" event={"ID":"15000f24-3a77-44b2-aa10-6e34b1608c69","Type":"ContainerDied","Data":"591fbc2cd30b70027bfcfcba1f470dc3a44f1b18b1328e6c4ec274b4ed22c576"}
Jul 24 23:40:23.385898 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:23.385563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m" event={"ID":"15000f24-3a77-44b2-aa10-6e34b1608c69","Type":"ContainerStarted","Data":"167e68449f58f786ee64f20144343ea93d85ee2751a82239980a4ae4bb9b41fd"}
Jul 24 23:40:24.538554 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:24.538528    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:24.697406 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:24.697312    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15000f24-3a77-44b2-aa10-6e34b1608c69-maas-api-service-ca\") pod \"15000f24-3a77-44b2-aa10-6e34b1608c69\" (UID: \"15000f24-3a77-44b2-aa10-6e34b1608c69\") "
Jul 24 23:40:24.697685 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:24.697662    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15000f24-3a77-44b2-aa10-6e34b1608c69-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15000f24-3a77-44b2-aa10-6e34b1608c69" (UID: "15000f24-3a77-44b2-aa10-6e34b1608c69"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:40:24.798148 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:24.798090    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15000f24-3a77-44b2-aa10-6e34b1608c69-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:40:25.398613 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:25.398585    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"
Jul 24 23:40:25.398789 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:25.398582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m" event={"ID":"15000f24-3a77-44b2-aa10-6e34b1608c69","Type":"ContainerDied","Data":"167e68449f58f786ee64f20144343ea93d85ee2751a82239980a4ae4bb9b41fd"}
Jul 24 23:40:25.398789 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:25.398724    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="167e68449f58f786ee64f20144343ea93d85ee2751a82239980a4ae4bb9b41fd"
Jul 24 23:40:36.394766 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:36.394729    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:40:37.533700 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:37.533593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:37.636263 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:37.636230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:37.735643 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:37.735597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:42.061460 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:42.061416    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"]
Jul 24 23:40:42.062832 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:42.062801    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:42.064353 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:42.064329    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0aa0e6-4g78m"]
Jul 24 23:40:42.443407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:42.443354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:43.695529 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:43.695498    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15000f24-3a77-44b2-aa10-6e34b1608c69" path="/var/lib/kubelet/pods/15000f24-3a77-44b2-aa10-6e34b1608c69/volumes"
Jul 24 23:40:47.099554 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:47.099514    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:40:47.427752 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:47.427664    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"]
Jul 24 23:40:47.475841 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:40:47.475810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:02.447526 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.447472    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b" podUID="80a2f455-0443-4745-9458-cca77b5560f5" containerName="main" containerID="cri-o://aa42dbaf4833d0ad1a8fa578e174afce55ff0ccd777a1c33b2855dcb7a9c4042" gracePeriod=45
Jul 24 23:41:02.637654 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.637613    2564 generic.go:356] "Generic (PLEG): container finished" podID="80a2f455-0443-4745-9458-cca77b5560f5" containerID="aa42dbaf4833d0ad1a8fa578e174afce55ff0ccd777a1c33b2855dcb7a9c4042" exitCode=0
Jul 24 23:41:02.637829 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.637683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b" event={"ID":"80a2f455-0443-4745-9458-cca77b5560f5","Type":"ContainerDied","Data":"aa42dbaf4833d0ad1a8fa578e174afce55ff0ccd777a1c33b2855dcb7a9c4042"}
Jul 24 23:41:02.720368 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.720344    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:41:02.870287 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870251    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-dshm\") pod \"80a2f455-0443-4745-9458-cca77b5560f5\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") "
Jul 24 23:41:02.870287 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870295    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-model-cache\") pod \"80a2f455-0443-4745-9458-cca77b5560f5\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") "
Jul 24 23:41:02.870571 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870351    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-home\") pod \"80a2f455-0443-4745-9458-cca77b5560f5\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") "
Jul 24 23:41:02.870571 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-tmp-dir\") pod \"80a2f455-0443-4745-9458-cca77b5560f5\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") "
Jul 24 23:41:02.870571 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870411    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2f455-0443-4745-9458-cca77b5560f5-tls-certs\") pod \"80a2f455-0443-4745-9458-cca77b5560f5\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") "
Jul 24 23:41:02.870571 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870438    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txcxm\" (UniqueName: \"kubernetes.io/projected/80a2f455-0443-4745-9458-cca77b5560f5-kube-api-access-txcxm\") pod \"80a2f455-0443-4745-9458-cca77b5560f5\" (UID: \"80a2f455-0443-4745-9458-cca77b5560f5\") "
Jul 24 23:41:02.870776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870675    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-home" (OuterVolumeSpecName: "home") pod "80a2f455-0443-4745-9458-cca77b5560f5" (UID: "80a2f455-0443-4745-9458-cca77b5560f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:02.870776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870690    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "80a2f455-0443-4745-9458-cca77b5560f5" (UID: "80a2f455-0443-4745-9458-cca77b5560f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:02.870776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870668    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80a2f455-0443-4745-9458-cca77b5560f5" (UID: "80a2f455-0443-4745-9458-cca77b5560f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:02.870880 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870789    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-home\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:02.870880 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870838    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-tmp-dir\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:02.870880 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.870852    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-model-cache\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:02.872738 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.872709    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-dshm" (OuterVolumeSpecName: "dshm") pod "80a2f455-0443-4745-9458-cca77b5560f5" (UID: "80a2f455-0443-4745-9458-cca77b5560f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:02.872849 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.872731    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a2f455-0443-4745-9458-cca77b5560f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80a2f455-0443-4745-9458-cca77b5560f5" (UID: "80a2f455-0443-4745-9458-cca77b5560f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:41:02.872849 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.872732    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a2f455-0443-4745-9458-cca77b5560f5-kube-api-access-txcxm" (OuterVolumeSpecName: "kube-api-access-txcxm") pod "80a2f455-0443-4745-9458-cca77b5560f5" (UID: "80a2f455-0443-4745-9458-cca77b5560f5"). InnerVolumeSpecName "kube-api-access-txcxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:41:02.971766 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.971672    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2f455-0443-4745-9458-cca77b5560f5-tls-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:02.971766 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.971710    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txcxm\" (UniqueName: \"kubernetes.io/projected/80a2f455-0443-4745-9458-cca77b5560f5-kube-api-access-txcxm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:02.971766 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:02.971719    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2f455-0443-4745-9458-cca77b5560f5-dshm\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:03.645510 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:03.645478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b" event={"ID":"80a2f455-0443-4745-9458-cca77b5560f5","Type":"ContainerDied","Data":"ee0bd50fa60da596f821e0cfd758380a83339d323fb68179e0e19f576890df6c"}
Jul 24 23:41:03.645510 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:03.645501    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"
Jul 24 23:41:03.646009 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:03.645526    2564 scope.go:117] "RemoveContainer" containerID="aa42dbaf4833d0ad1a8fa578e174afce55ff0ccd777a1c33b2855dcb7a9c4042"
Jul 24 23:41:03.668730 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:03.668696    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"]
Jul 24 23:41:03.675222 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:03.675194    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-0aa0e6/ipp-route-0aa0e6-kserve-766d9f9f99-b4p7b"]
Jul 24 23:41:03.696302 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:03.696269    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a2f455-0443-4745-9458-cca77b5560f5" path="/var/lib/kubelet/pods/80a2f455-0443-4745-9458-cca77b5560f5/volumes"
Jul 24 23:41:24.188600 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:24.188568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:24.431946 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:24.431912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:29.086001 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:29.085960    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:30.083544 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:30.083510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:35.566080 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:35.566044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:35.639334 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:35.639303    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:35.834974 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:35.834940    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"]
Jul 24 23:41:35.835266 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:35.835221    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy" containerID="cri-o://824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7" gracePeriod=30
Jul 24 23:41:35.857639 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:35.857595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:36.092894 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.092812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:36.212435 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212403    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"]
Jul 24 23:41:36.212883 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212871    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a2f455-0443-4745-9458-cca77b5560f5" containerName="main"
Jul 24 23:41:36.212927 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212885    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a2f455-0443-4745-9458-cca77b5560f5" containerName="main"
Jul 24 23:41:36.212927 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212894    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15000f24-3a77-44b2-aa10-6e34b1608c69" containerName="revoke-keys"
Jul 24 23:41:36.212927 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212900    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15000f24-3a77-44b2-aa10-6e34b1608c69" containerName="revoke-keys"
Jul 24 23:41:36.213035 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212973    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a2f455-0443-4745-9458-cca77b5560f5" containerName="main"
Jul 24 23:41:36.213035 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.212986    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15000f24-3a77-44b2-aa10-6e34b1608c69" containerName="revoke-keys"
Jul 24 23:41:36.216208 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.216192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:36.218817 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.218788    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:41:36.221958 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.221935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"]
Jul 24 23:41:36.292545 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.292513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b808f59-8f0a-4a87-9421-f4a9bdef72f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv\" (UID: \"0b808f59-8f0a-4a87-9421-f4a9bdef72f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:36.393194 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.393094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b808f59-8f0a-4a87-9421-f4a9bdef72f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv\" (UID: \"0b808f59-8f0a-4a87-9421-f4a9bdef72f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:36.393716 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.393696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b808f59-8f0a-4a87-9421-f4a9bdef72f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv\" (UID: \"0b808f59-8f0a-4a87-9421-f4a9bdef72f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:36.529670 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.529632    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:36.672884 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.672854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"]
Jul 24 23:41:36.674574 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:41:36.674550    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b808f59_8f0a_4a87_9421_f4a9bdef72f7.slice/crio-24c1271f15aad0e29b20c37d6f36aba6ad7668493642d3427c0d5c50ece61c4a WatchSource:0}: Error finding container 24c1271f15aad0e29b20c37d6f36aba6ad7668493642d3427c0d5c50ece61c4a: Status 404 returned error can't find the container with id 24c1271f15aad0e29b20c37d6f36aba6ad7668493642d3427c0d5c50ece61c4a
Jul 24 23:41:36.848195 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:36.848162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv" event={"ID":"0b808f59-8f0a-4a87-9421-f4a9bdef72f7","Type":"ContainerStarted","Data":"24c1271f15aad0e29b20c37d6f36aba6ad7668493642d3427c0d5c50ece61c4a"}
Jul 24 23:41:37.856385 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:37.856351    2564 generic.go:356] "Generic (PLEG): container finished" podID="0b808f59-8f0a-4a87-9421-f4a9bdef72f7" containerID="9b32e35dad4ddf572e2a1f330e686b4874961fd0fbc12bdb327c0d0860f1c3de" exitCode=0
Jul 24 23:41:37.856831 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:37.856405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv" event={"ID":"0b808f59-8f0a-4a87-9421-f4a9bdef72f7","Type":"ContainerDied","Data":"9b32e35dad4ddf572e2a1f330e686b4874961fd0fbc12bdb327c0d0860f1c3de"}
Jul 24 23:41:39.006509 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.006484    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:39.121732 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.121644    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b808f59-8f0a-4a87-9421-f4a9bdef72f7-maas-api-service-ca\") pod \"0b808f59-8f0a-4a87-9421-f4a9bdef72f7\" (UID: \"0b808f59-8f0a-4a87-9421-f4a9bdef72f7\") "
Jul 24 23:41:39.122020 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.121997    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b808f59-8f0a-4a87-9421-f4a9bdef72f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b808f59-8f0a-4a87-9421-f4a9bdef72f7" (UID: "0b808f59-8f0a-4a87-9421-f4a9bdef72f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:41:39.223402 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.223365    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b808f59-8f0a-4a87-9421-f4a9bdef72f7-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:39.869671 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.869640    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"
Jul 24 23:41:39.869868 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.869639    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv" event={"ID":"0b808f59-8f0a-4a87-9421-f4a9bdef72f7","Type":"ContainerDied","Data":"24c1271f15aad0e29b20c37d6f36aba6ad7668493642d3427c0d5c50ece61c4a"}
Jul 24 23:41:39.869868 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:39.869762    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24c1271f15aad0e29b20c37d6f36aba6ad7668493642d3427c0d5c50ece61c4a"
Jul 24 23:41:41.099003 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.098938    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:41:41.244296 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244218    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znqx4\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-kube-api-access-znqx4\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244296 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244290    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-podinfo\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244521 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244331    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-certs\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244521 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-envoy\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244521 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244390    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-token\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244521 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244417    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-data\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244737 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244558    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-credential-socket\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244737 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244594    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-socket\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244737 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244617    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istiod-ca-cert\") pod \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\" (UID: \"37bdf660-ff3b-4609-a83d-85d9e83e3e5e\") "
Jul 24 23:41:41.244737 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:41.244964 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244883    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:41.244964 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244925    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:41.244964 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244952    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-socket\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.245157 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.244972    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-workload-certs\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.245248 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.245226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:41:41.245960 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.245926    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-data" (OuterVolumeSpecName: "istio-data") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:41.246559 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.246534    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-kube-api-access-znqx4" (OuterVolumeSpecName: "kube-api-access-znqx4") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "kube-api-access-znqx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:41:41.246682 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.246664    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:41:41.246779 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.246759    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-token" (OuterVolumeSpecName: "istio-token") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:41:41.246950 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.246931    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37bdf660-ff3b-4609-a83d-85d9e83e3e5e" (UID: "37bdf660-ff3b-4609-a83d-85d9e83e3e5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 23:41:41.346016 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.345962    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-token\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.346016 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.346007    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-data\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.346016 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.346016    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-credential-socket\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.346016 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.346025    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istiod-ca-cert\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.346016 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.346033    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znqx4\" (UniqueName: \"kubernetes.io/projected/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-kube-api-access-znqx4\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.346374 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.346042    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-podinfo\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.346374 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.346050    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bdf660-ff3b-4609-a83d-85d9e83e3e5e-istio-envoy\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:41.883324 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.883287    2564 generic.go:356] "Generic (PLEG): container finished" podID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerID="824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7" exitCode=0
Jul 24 23:41:41.883530 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.883341    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"
Jul 24 23:41:41.883530 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.883363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" event={"ID":"37bdf660-ff3b-4609-a83d-85d9e83e3e5e","Type":"ContainerDied","Data":"824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7"}
Jul 24 23:41:41.883530 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.883424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8" event={"ID":"37bdf660-ff3b-4609-a83d-85d9e83e3e5e","Type":"ContainerDied","Data":"3099939d96adc49fccb7136199b0c93d8f7758d9c39bbe3fdbb12a53ecb3f032"}
Jul 24 23:41:41.883530 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.883452    2564 scope.go:117] "RemoveContainer" containerID="824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7"
Jul 24 23:41:41.893257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.893238    2564 scope.go:117] "RemoveContainer" containerID="82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"
Jul 24 23:41:41.903954 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.903742    2564 scope.go:117] "RemoveContainer" containerID="824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7"
Jul 24 23:41:41.904117 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:41:41.904093    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7\": container with ID starting with 824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7 not found: ID does not exist" containerID="824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7"
Jul 24 23:41:41.904207 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.904174    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7"} err="failed to get container status \"824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7\": rpc error: code = NotFound desc = could not find container \"824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7\": container with ID starting with 824a2af70f054be9fac911c5603f4295cc99b0f3c6ba1104875db856d6585fb7 not found: ID does not exist"
Jul 24 23:41:41.904207 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.904202    2564 scope.go:117] "RemoveContainer" containerID="82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"
Jul 24 23:41:41.904416 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.904396    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"]
Jul 24 23:41:41.904495 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:41:41.904451    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de\": container with ID starting with 82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de not found: ID does not exist" containerID="82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"
Jul 24 23:41:41.904543 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.904483    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de"} err="failed to get container status \"82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de\": rpc error: code = NotFound desc = could not find container \"82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de\": container with ID starting with 82fdeadf52a5ce453d713c0e9b891a4631578a42bfaaac04611f67dd849048de not found: ID does not exist"
Jul 24 23:41:41.907000 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:41.906982    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-0aa0e6-openshift-default-55fdd9f688-bcdx8"]
Jul 24 23:41:43.695673 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:43.695640    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" path="/var/lib/kubelet/pods/37bdf660-ff3b-4609-a83d-85d9e83e3e5e/volumes"
Jul 24 23:41:46.603610 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.603573    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"]
Jul 24 23:41:46.604111 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.603820    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2" podUID="2f19712a-5771-45a2-aee2-ffb901f0a00e" containerName="maas-api" containerID="cri-o://2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d" gracePeriod=30
Jul 24 23:41:46.631206 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.631175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:46.730632 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.730579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:46.782071 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.782036    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:46.880821 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.880798    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:41:46.916488 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.916450    2564 generic.go:356] "Generic (PLEG): container finished" podID="2f19712a-5771-45a2-aee2-ffb901f0a00e" containerID="2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d" exitCode=0
Jul 24 23:41:46.916660 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.916596    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"
Jul 24 23:41:46.916723 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.916660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2" event={"ID":"2f19712a-5771-45a2-aee2-ffb901f0a00e","Type":"ContainerDied","Data":"2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d"}
Jul 24 23:41:46.916773 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.916739    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2" event={"ID":"2f19712a-5771-45a2-aee2-ffb901f0a00e","Type":"ContainerDied","Data":"26a49eb91064f36d69fef70063fdc52676fbff64e629f047fa579ebe49125ec6"}
Jul 24 23:41:46.916773 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.916766    2564 scope.go:117] "RemoveContainer" containerID="2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d"
Jul 24 23:41:46.930845 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.930823    2564 scope.go:117] "RemoveContainer" containerID="2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d"
Jul 24 23:41:46.931188 ip-10-0-143-78 kubenswrapper[2564]: E0724 23:41:46.931165    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d\": container with ID starting with 2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d not found: ID does not exist" containerID="2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d"
Jul 24 23:41:46.931286 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:46.931197    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d"} err="failed to get container status \"2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d\": rpc error: code = NotFound desc = could not find container \"2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d\": container with ID starting with 2cb8b0f324c7ac0ca4eeae09098fa8dcd1dd66d7149ab3782802f0e4f19b677d not found: ID does not exist"
Jul 24 23:41:47.007270 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.007238    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dk5v7\" (UniqueName: \"kubernetes.io/projected/2f19712a-5771-45a2-aee2-ffb901f0a00e-kube-api-access-dk5v7\") pod \"2f19712a-5771-45a2-aee2-ffb901f0a00e\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") "
Jul 24 23:41:47.007448 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.007282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f19712a-5771-45a2-aee2-ffb901f0a00e-maas-api-tls\") pod \"2f19712a-5771-45a2-aee2-ffb901f0a00e\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") "
Jul 24 23:41:47.007448 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.007383    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f19712a-5771-45a2-aee2-ffb901f0a00e-openshift-service-ca\") pod \"2f19712a-5771-45a2-aee2-ffb901f0a00e\" (UID: \"2f19712a-5771-45a2-aee2-ffb901f0a00e\") "
Jul 24 23:41:47.007726 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.007702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f19712a-5771-45a2-aee2-ffb901f0a00e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f19712a-5771-45a2-aee2-ffb901f0a00e" (UID: "2f19712a-5771-45a2-aee2-ffb901f0a00e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:41:47.009505 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.009483    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f19712a-5771-45a2-aee2-ffb901f0a00e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f19712a-5771-45a2-aee2-ffb901f0a00e" (UID: "2f19712a-5771-45a2-aee2-ffb901f0a00e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:41:47.009627 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.009607    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f19712a-5771-45a2-aee2-ffb901f0a00e-kube-api-access-dk5v7" (OuterVolumeSpecName: "kube-api-access-dk5v7") pod "2f19712a-5771-45a2-aee2-ffb901f0a00e" (UID: "2f19712a-5771-45a2-aee2-ffb901f0a00e"). InnerVolumeSpecName "kube-api-access-dk5v7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:41:47.108328 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.108251    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk5v7\" (UniqueName: \"kubernetes.io/projected/2f19712a-5771-45a2-aee2-ffb901f0a00e-kube-api-access-dk5v7\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:47.108328 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.108278    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f19712a-5771-45a2-aee2-ffb901f0a00e-maas-api-tls\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:47.108328 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.108287    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f19712a-5771-45a2-aee2-ffb901f0a00e-openshift-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:41:47.241794 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.241754    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"]
Jul 24 23:41:47.245968 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.245939    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3fb847-7cf49d69cd-dtsf2"]
Jul 24 23:41:47.695580 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:47.695548    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f19712a-5771-45a2-aee2-ffb901f0a00e" path="/var/lib/kubelet/pods/2f19712a-5771-45a2-aee2-ffb901f0a00e/volumes"
Jul 24 23:41:49.791977 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:49.791941    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:41:51.284240 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:51.284202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:51.286116 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:51.286090    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"]
Jul 24 23:41:51.288805 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:51.288784    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3fb847-hm8qv"]
Jul 24 23:41:51.672865 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:51.672832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:51.695903 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:51.695877    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b808f59-8f0a-4a87-9421-f4a9bdef72f7" path="/var/lib/kubelet/pods/0b808f59-8f0a-4a87-9421-f4a9bdef72f7/volumes"
Jul 24 23:41:57.435134 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:57.435094    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:57.501911 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:57.501876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:57.722392 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:57.722302    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:57.954793 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:57.954761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:58.101818 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.101783    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"]
Jul 24 23:41:58.102221 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102206    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f19712a-5771-45a2-aee2-ffb901f0a00e" containerName="maas-api"
Jul 24 23:41:58.102271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102223    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f19712a-5771-45a2-aee2-ffb901f0a00e" containerName="maas-api"
Jul 24 23:41:58.102271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102233    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy"
Jul 24 23:41:58.102271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102240    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy"
Jul 24 23:41:58.102271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102254    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b808f59-8f0a-4a87-9421-f4a9bdef72f7" containerName="revoke-keys"
Jul 24 23:41:58.102271 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102260    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b808f59-8f0a-4a87-9421-f4a9bdef72f7" containerName="revoke-keys"
Jul 24 23:41:58.102433 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102339    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy"
Jul 24 23:41:58.102433 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102346    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b808f59-8f0a-4a87-9421-f4a9bdef72f7" containerName="revoke-keys"
Jul 24 23:41:58.102433 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.102353    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f19712a-5771-45a2-aee2-ffb901f0a00e" containerName="maas-api"
Jul 24 23:41:58.106791 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.106775    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:41:58.109536 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.109513    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:41:58.109738 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.109514    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:41:58.112166 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.112142    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"]
Jul 24 23:41:58.215031 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.215000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q\" (UID: \"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:41:58.316380 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.316342    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q\" (UID: \"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:41:58.316987 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.316963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q\" (UID: \"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:41:58.420022 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.419931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:41:58.559219 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.559183    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"]
Jul 24 23:41:58.559616 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:41:58.559486    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d1cf63_cb20_4b62_8f58_7fbb50a3d4a5.slice/crio-585fc335ea4e98f723e54dd30ed479716697fb62ab376f4872fe1bd1baf01e7f WatchSource:0}: Error finding container 585fc335ea4e98f723e54dd30ed479716697fb62ab376f4872fe1bd1baf01e7f: Status 404 returned error can't find the container with id 585fc335ea4e98f723e54dd30ed479716697fb62ab376f4872fe1bd1baf01e7f
Jul 24 23:41:58.993578 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.993538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q" event={"ID":"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5","Type":"ContainerStarted","Data":"001bc8d60e726d7d2eccd4bedd467e742211868bfc07e69263d21ae53abd6e83"}
Jul 24 23:41:58.993775 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:58.993625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q" event={"ID":"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5","Type":"ContainerStarted","Data":"585fc335ea4e98f723e54dd30ed479716697fb62ab376f4872fe1bd1baf01e7f"}
Jul 24 23:41:59.009983 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:41:59.009872    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q" podStartSLOduration=1.00985626 podStartE2EDuration="1.00985626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:41:59.007277988 +0000 UTC m=+2979.860604529" watchObservedRunningTime="2026-07-24 23:41:59.00985626 +0000 UTC m=+2979.863182778"
Jul 24 23:42:00.001489 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:00.001452    2564 generic.go:356] "Generic (PLEG): container finished" podID="51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5" containerID="001bc8d60e726d7d2eccd4bedd467e742211868bfc07e69263d21ae53abd6e83" exitCode=0
Jul 24 23:42:00.001940 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:00.001540    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q" event={"ID":"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5","Type":"ContainerDied","Data":"001bc8d60e726d7d2eccd4bedd467e742211868bfc07e69263d21ae53abd6e83"}
Jul 24 23:42:01.144472 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:01.144446    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:42:01.243233 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:01.243189    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5-maas-api-service-ca\") pod \"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5\" (UID: \"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5\") "
Jul 24 23:42:01.243585 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:01.243560    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5" (UID: "51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:01.344758 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:01.344716    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:42:02.014620 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:02.014529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q" event={"ID":"51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5","Type":"ContainerDied","Data":"585fc335ea4e98f723e54dd30ed479716697fb62ab376f4872fe1bd1baf01e7f"}
Jul 24 23:42:02.014620 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:02.014580    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="585fc335ea4e98f723e54dd30ed479716697fb62ab376f4872fe1bd1baf01e7f"
Jul 24 23:42:02.014620 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:02.014546    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"
Jul 24 23:42:08.880162 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:08.880108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:08.951771 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:08.951740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:09.008008 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:09.007968    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:12.192605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:12.192559    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:42:13.168248 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.168209    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"]
Jul 24 23:42:13.172992 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.172960    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe475f-mns7q"]
Jul 24 23:42:13.174389 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.174358    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:13.406062 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.406028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:13.461251 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.461167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:13.559380 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.559346    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:13.696231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:13.696197    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5" path="/var/lib/kubelet/pods/51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5/volumes"
Jul 24 23:42:19.363407 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.363372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:19.464419 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.464387    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:19.640084 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.640008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:19.852111 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.852082    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:19.888174 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.888117    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:42:19.901248 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.901185    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:42:19.974399 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974359    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"]
Jul 24 23:42:19.974872 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974855    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5" containerName="revoke-keys"
Jul 24 23:42:19.974918 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974874    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5" containerName="revoke-keys"
Jul 24 23:42:19.974918 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974892    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy"
Jul 24 23:42:19.974918 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974898    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy"
Jul 24 23:42:19.975032 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974964    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d1cf63-cb20-4b62-8f58-7fbb50a3d4a5" containerName="revoke-keys"
Jul 24 23:42:19.975032 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.974973    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37bdf660-ff3b-4609-a83d-85d9e83e3e5e" containerName="istio-proxy"
Jul 24 23:42:19.978360 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.978344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:19.980898 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.980874    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:42:19.981029 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.980874    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:42:19.987579 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:19.987556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"]
Jul 24 23:42:20.127451 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:20.127422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b57a620-7a8a-4283-857c-2aec2b425914-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk\" (UID: \"5b57a620-7a8a-4283-857c-2aec2b425914\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:20.228624 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:20.228546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b57a620-7a8a-4283-857c-2aec2b425914-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk\" (UID: \"5b57a620-7a8a-4283-857c-2aec2b425914\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:20.229201 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:20.229184    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b57a620-7a8a-4283-857c-2aec2b425914-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk\" (UID: \"5b57a620-7a8a-4283-857c-2aec2b425914\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:20.289848 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:20.289821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:20.427222 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:20.427192    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"]
Jul 24 23:42:20.428629 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:42:20.428600    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b57a620_7a8a_4283_857c_2aec2b425914.slice/crio-2caac7b76e332cbab51176e32442eaa9d958702392098e7c950126ba2dcbda0c WatchSource:0}: Error finding container 2caac7b76e332cbab51176e32442eaa9d958702392098e7c950126ba2dcbda0c: Status 404 returned error can't find the container with id 2caac7b76e332cbab51176e32442eaa9d958702392098e7c950126ba2dcbda0c
Jul 24 23:42:21.131530 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:21.131489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk" event={"ID":"5b57a620-7a8a-4283-857c-2aec2b425914","Type":"ContainerStarted","Data":"876d70263c0cd6ce0b8ad2bfa3526f7e89734b4d0ea9980ac34d92ac0079fe32"}
Jul 24 23:42:21.131736 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:21.131554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk" event={"ID":"5b57a620-7a8a-4283-857c-2aec2b425914","Type":"ContainerStarted","Data":"2caac7b76e332cbab51176e32442eaa9d958702392098e7c950126ba2dcbda0c"}
Jul 24 23:42:21.147867 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:21.147823    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk" podStartSLOduration=2.147807318 podStartE2EDuration="2.147807318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:42:21.146079981 +0000 UTC m=+3001.999406543" watchObservedRunningTime="2026-07-24 23:42:21.147807318 +0000 UTC m=+3002.001133880"
Jul 24 23:42:22.139486 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:22.139447    2564 generic.go:356] "Generic (PLEG): container finished" podID="5b57a620-7a8a-4283-857c-2aec2b425914" containerID="876d70263c0cd6ce0b8ad2bfa3526f7e89734b4d0ea9980ac34d92ac0079fe32" exitCode=0
Jul 24 23:42:22.139976 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:22.139535    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk" event={"ID":"5b57a620-7a8a-4283-857c-2aec2b425914","Type":"ContainerDied","Data":"876d70263c0cd6ce0b8ad2bfa3526f7e89734b4d0ea9980ac34d92ac0079fe32"}
Jul 24 23:42:23.298266 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:23.298242    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:23.454791 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:23.454704    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b57a620-7a8a-4283-857c-2aec2b425914-maas-api-service-ca\") pod \"5b57a620-7a8a-4283-857c-2aec2b425914\" (UID: \"5b57a620-7a8a-4283-857c-2aec2b425914\") "
Jul 24 23:42:23.455087 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:23.455067    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b57a620-7a8a-4283-857c-2aec2b425914-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b57a620-7a8a-4283-857c-2aec2b425914" (UID: "5b57a620-7a8a-4283-857c-2aec2b425914"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:23.555877 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:23.555836    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b57a620-7a8a-4283-857c-2aec2b425914-maas-api-service-ca\") on node \"ip-10-0-143-78.ec2.internal\" DevicePath \"\""
Jul 24 23:42:24.153581 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:24.153547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk" event={"ID":"5b57a620-7a8a-4283-857c-2aec2b425914","Type":"ContainerDied","Data":"2caac7b76e332cbab51176e32442eaa9d958702392098e7c950126ba2dcbda0c"}
Jul 24 23:42:24.153581 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:24.153585    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"
Jul 24 23:42:24.153800 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:24.153604    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2caac7b76e332cbab51176e32442eaa9d958702392098e7c950126ba2dcbda0c"
Jul 24 23:42:30.496944 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:30.496914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:30.597816 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:30.597786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:30.697263 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:30.697226    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:33.391809 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:33.391774    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hgjfj"]
Jul 24 23:42:35.040227 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:35.040187    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"]
Jul 24 23:42:35.041306 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:35.041273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:35.042907 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:35.042884    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6f3ce2-2n9wk"]
Jul 24 23:42:35.425922 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:35.425887    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:35.695185 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:35.695092    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b57a620-7a8a-4283-857c-2aec2b425914" path="/var/lib/kubelet/pods/5b57a620-7a8a-4283-857c-2aec2b425914/volumes"
Jul 24 23:42:41.142181 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:41.142149    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:41.199001 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:41.198967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:41.411840 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:41.411754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:41.645098 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:41.645060    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:42.085439 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:42.085407    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5668767cc9-gqqnh_8ded3901-d129-4ac2-b2fb-998ee9b7b794/authorino/0.log"
Jul 24 23:42:47.507013 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:47.506980    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jr2bk_bc715d7e-142f-4c70-a2c9-86cbd9077540/manager/0.log"
Jul 24 23:42:47.626636 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:47.626607    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r4q4f_95265a02-5c3d-4466-90cb-d295ef074376/manager/0.log"
Jul 24 23:42:47.880744 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:47.880721    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z79ww_3185626a-2701-4b83-9807-58def62c683f/server/0.log"
Jul 24 23:42:48.002546 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:48.002519    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vthzr_a36ff974-d2d8-4de7-ac17-b7885415da66/manager/0.log"
Jul 24 23:42:48.150160 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:48.150060    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:48.388663 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:48.388632    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-qztx5_f26aa5f2-483b-449d-a43a-c018b9c75ebc/copy-manifests/0.log"
Jul 24 23:42:48.395522 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:48.395498    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-qztx5_f26aa5f2-483b-449d-a43a-c018b9c75ebc/manager/0.log"
Jul 24 23:42:49.864490 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:49.864461    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7_90698fdb-a1f9-45ac-99f5-3a08112ac645/util/0.log"
Jul 24 23:42:49.871938 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:49.871913    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7_90698fdb-a1f9-45ac-99f5-3a08112ac645/pull/0.log"
Jul 24 23:42:49.879615 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:49.879591    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7_90698fdb-a1f9-45ac-99f5-3a08112ac645/extract/0.log"
Jul 24 23:42:49.995703 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:49.995672    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz_02959104-3c14-4720-b78e-040bea03526d/util/0.log"
Jul 24 23:42:50.002387 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.002359    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz_02959104-3c14-4720-b78e-040bea03526d/pull/0.log"
Jul 24 23:42:50.009033 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.009003    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz_02959104-3c14-4720-b78e-040bea03526d/extract/0.log"
Jul 24 23:42:50.127043 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.126949    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9_f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6/pull/0.log"
Jul 24 23:42:50.134288 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.134268    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9_f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6/extract/0.log"
Jul 24 23:42:50.140557 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.140541    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9_f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6/util/0.log"
Jul 24 23:42:50.256218 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.256189    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb_7bb24c70-ab49-4285-b359-c278085e63e4/util/0.log"
Jul 24 23:42:50.263028 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.263007    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb_7bb24c70-ab49-4285-b359-c278085e63e4/pull/0.log"
Jul 24 23:42:50.269289 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.269269    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb_7bb24c70-ab49-4285-b359-c278085e63e4/extract/0.log"
Jul 24 23:42:50.387942 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.387866    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5668767cc9-gqqnh_8ded3901-d129-4ac2-b2fb-998ee9b7b794/authorino/0.log"
Jul 24 23:42:50.505421 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.505392    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6vn45_a4312f4f-93df-4d70-91fc-bec77b2ba638/manager/0.log"
Jul 24 23:42:50.620316 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.620292    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dzbwf_18353da9-5b95-4d28-acce-c31238b94625/manager/0.log"
Jul 24 23:42:50.849531 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.849501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lvbvc_472a249b-9d6a-4999-b9b8-91a0d0ea31d9/registry-server/0.log"
Jul 24 23:42:50.968295 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:50.968270    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sppcg_dc7f209e-c378-484d-83e5-b5f48097914b/manager/0.log"
Jul 24 23:42:51.082779 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:51.082745    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hgjfj_31fffaa6-f925-4523-bd3b-369d1fd3d6c3/limitador/0.log"
Jul 24 23:42:51.199553 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:51.199475    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kq62x_4fe88e94-84bd-4bba-b242-c19cbb0d6467/manager/0.log"
Jul 24 23:42:51.561361 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:51.561285    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft_2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad/istio-proxy/0.log"
Jul 24 23:42:51.934655 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:51.934624    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-bt5q9_d22bdb04-157a-4ccd-bab8-c672428aada2/kube-auth-proxy/0.log"
Jul 24 23:42:52.174355 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:52.174325    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kc9w5_b244bf57-7ebf-4e84-8661-5ce003e1bfe3/payload-pre-processing/0.log"
Jul 24 23:42:53.206200 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:53.206170    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gr6dx_17af7937-9522-4680-967b-755f369a83f9/main/0.log"
Jul 24 23:42:58.788350 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.788321    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jpvfn/must-gather-h4bnm"]
Jul 24 23:42:58.788787 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.788743    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b57a620-7a8a-4283-857c-2aec2b425914" containerName="revoke-keys"
Jul 24 23:42:58.788787 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.788754    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b57a620-7a8a-4283-857c-2aec2b425914" containerName="revoke-keys"
Jul 24 23:42:58.788863 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.788825    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b57a620-7a8a-4283-857c-2aec2b425914" containerName="revoke-keys"
Jul 24 23:42:58.793598 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.793578    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:58.796550 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.796530    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jpvfn\"/\"default-dockercfg-br4hs\""
Jul 24 23:42:58.797329 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.797314    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jpvfn\"/\"kube-root-ca.crt\""
Jul 24 23:42:58.797589 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.797572    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jpvfn\"/\"openshift-service-ca.crt\""
Jul 24 23:42:58.801871 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.801729    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jpvfn/must-gather-h4bnm"]
Jul 24 23:42:58.803195 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.803113    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:58.864907 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.864864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrvzq\" (UniqueName: \"kubernetes.io/projected/ef13ee9e-fa59-45d0-9888-2f78a6ac6f03-kube-api-access-hrvzq\") pod \"must-gather-h4bnm\" (UID: \"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03\") " pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:58.865085 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.864921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef13ee9e-fa59-45d0-9888-2f78a6ac6f03-must-gather-output\") pod \"must-gather-h4bnm\" (UID: \"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03\") " pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:58.898984 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.898956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:58.965605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.965572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrvzq\" (UniqueName: \"kubernetes.io/projected/ef13ee9e-fa59-45d0-9888-2f78a6ac6f03-kube-api-access-hrvzq\") pod \"must-gather-h4bnm\" (UID: \"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03\") " pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:58.965771 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.965617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef13ee9e-fa59-45d0-9888-2f78a6ac6f03-must-gather-output\") pod \"must-gather-h4bnm\" (UID: \"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03\") " pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:58.965916 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.965902    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef13ee9e-fa59-45d0-9888-2f78a6ac6f03-must-gather-output\") pod \"must-gather-h4bnm\" (UID: \"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03\") " pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:58.978563 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:58.978540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrvzq\" (UniqueName: \"kubernetes.io/projected/ef13ee9e-fa59-45d0-9888-2f78a6ac6f03-kube-api-access-hrvzq\") pod \"must-gather-h4bnm\" (UID: \"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03\") " pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:59.104909 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:59.104877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jpvfn/must-gather-h4bnm"
Jul 24 23:42:59.242559 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:59.242533    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jpvfn/must-gather-h4bnm"]
Jul 24 23:42:59.244517 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:42:59.244490    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef13ee9e_fa59_45d0_9888_2f78a6ac6f03.slice/crio-1e8a8f643bf6183ff4d97717d558fc122725dcbb8d40bcad78dca8ce53f6469b WatchSource:0}: Error finding container 1e8a8f643bf6183ff4d97717d558fc122725dcbb8d40bcad78dca8ce53f6469b: Status 404 returned error can't find the container with id 1e8a8f643bf6183ff4d97717d558fc122725dcbb8d40bcad78dca8ce53f6469b
Jul 24 23:42:59.246422 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:59.246407    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:42:59.371776 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:42:59.371688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/must-gather-h4bnm" event={"ID":"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03","Type":"ContainerStarted","Data":"1e8a8f643bf6183ff4d97717d558fc122725dcbb8d40bcad78dca8ce53f6469b"}
Jul 24 23:43:00.384211 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:00.384169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/must-gather-h4bnm" event={"ID":"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03","Type":"ContainerStarted","Data":"9a24931d7073bff65db9e7bdcf03decf82c371f2c8f99d2cd52989ad7dfaef35"}
Jul 24 23:43:00.384703 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:00.384228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/must-gather-h4bnm" event={"ID":"ef13ee9e-fa59-45d0-9888-2f78a6ac6f03","Type":"ContainerStarted","Data":"afcd708e319bde719baaac8339bd2313e128542df51797ab4aab4f38cae5a7d5"}
Jul 24 23:43:00.400045 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:00.399920    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jpvfn/must-gather-h4bnm" podStartSLOduration=1.645216759 podStartE2EDuration="2.399899086s" totalImagesPullingTime="754.682327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:43:00.399513117 +0000 UTC m=+3041.252839660" watchObservedRunningTime="2026-07-24 23:43:00.399899086 +0000 UTC m=+3041.253225628"
Jul 24 23:43:01.570732 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:01.570697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4mrph_7e5143b3-55f1-4714-8a04-a2d8f581b307/global-pull-secret-syncer/0.log"
Jul 24 23:43:01.708562 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:01.708532    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-68qcj_0614a849-84b5-44da-b548-40e21080bfd7/konnectivity-agent/0.log"
Jul 24 23:43:01.879845 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:01.879812    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-78.ec2.internal_62d620f13b2ef8b28e0eac26122212df/haproxy/0.log"
Jul 24 23:43:06.243065 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.243037    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7_90698fdb-a1f9-45ac-99f5-3a08112ac645/extract/0.log"
Jul 24 23:43:06.266168 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.266141    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7_90698fdb-a1f9-45ac-99f5-3a08112ac645/util/0.log"
Jul 24 23:43:06.287922 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.287897    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gfls7_90698fdb-a1f9-45ac-99f5-3a08112ac645/pull/0.log"
Jul 24 23:43:06.318593 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.318499    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz_02959104-3c14-4720-b78e-040bea03526d/extract/0.log"
Jul 24 23:43:06.357360 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.357330    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz_02959104-3c14-4720-b78e-040bea03526d/util/0.log"
Jul 24 23:43:06.388565 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.388539    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d7rvz_02959104-3c14-4720-b78e-040bea03526d/pull/0.log"
Jul 24 23:43:06.433246 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.433214    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9_f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6/extract/0.log"
Jul 24 23:43:06.483399 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.483343    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9_f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6/util/0.log"
Jul 24 23:43:06.508360 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.508257    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m45d9_f76b97e5-dad6-4ea0-bba7-e7b2db4d78d6/pull/0.log"
Jul 24 23:43:06.542925 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.542899    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb_7bb24c70-ab49-4285-b359-c278085e63e4/extract/0.log"
Jul 24 23:43:06.563802 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.563776    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb_7bb24c70-ab49-4285-b359-c278085e63e4/util/0.log"
Jul 24 23:43:06.583891 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:06.583866    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cqvtb_7bb24c70-ab49-4285-b359-c278085e63e4/pull/0.log"
Jul 24 23:43:07.050054 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.050010    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5668767cc9-gqqnh_8ded3901-d129-4ac2-b2fb-998ee9b7b794/authorino/0.log"
Jul 24 23:43:07.079428 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.079346    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6vn45_a4312f4f-93df-4d70-91fc-bec77b2ba638/manager/0.log"
Jul 24 23:43:07.112061 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.112025    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dzbwf_18353da9-5b95-4d28-acce-c31238b94625/manager/0.log"
Jul 24 23:43:07.196538 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.196501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lvbvc_472a249b-9d6a-4999-b9b8-91a0d0ea31d9/registry-server/0.log"
Jul 24 23:43:07.396092 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.395114    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sppcg_dc7f209e-c378-484d-83e5-b5f48097914b/manager/0.log"
Jul 24 23:43:07.417548 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.417516    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hgjfj_31fffaa6-f925-4523-bd3b-369d1fd3d6c3/limitador/0.log"
Jul 24 23:43:07.506577 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:07.505393    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kq62x_4fe88e94-84bd-4bba-b242-c19cbb0d6467/manager/0.log"
Jul 24 23:43:09.353882 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.353851    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/alertmanager/0.log"
Jul 24 23:43:09.378102 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.378076    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/config-reloader/0.log"
Jul 24 23:43:09.401583 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.401559    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/kube-rbac-proxy-web/0.log"
Jul 24 23:43:09.424294 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.424207    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/kube-rbac-proxy/0.log"
Jul 24 23:43:09.448081 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.447998    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/kube-rbac-proxy-metric/0.log"
Jul 24 23:43:09.469986 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.469960    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/prom-label-proxy/0.log"
Jul 24 23:43:09.491414 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.491376    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dff6dd2-d2d5-442a-b4a0-d7a516a0cde5/init-config-reloader/0.log"
Jul 24 23:43:09.544314 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.544279    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-h2xp6_f7bcc154-1403-44bb-99b5-b0d2c7d23168/cluster-monitoring-operator/0.log"
Jul 24 23:43:09.570812 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.570780    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4v4mn_8ef29862-0e6d-439c-8b64-f26d3f42e096/kube-state-metrics/0.log"
Jul 24 23:43:09.592743 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.592711    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4v4mn_8ef29862-0e6d-439c-8b64-f26d3f42e096/kube-rbac-proxy-main/0.log"
Jul 24 23:43:09.616064 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.615981    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4v4mn_8ef29862-0e6d-439c-8b64-f26d3f42e096/kube-rbac-proxy-self/0.log"
Jul 24 23:43:09.780765 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.780713    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qhrdg_7f9141c0-c865-4c73-9805-09e3fba95010/node-exporter/0.log"
Jul 24 23:43:09.800887 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.800846    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qhrdg_7f9141c0-c865-4c73-9805-09e3fba95010/kube-rbac-proxy/0.log"
Jul 24 23:43:09.826509 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.826425    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qhrdg_7f9141c0-c865-4c73-9805-09e3fba95010/init-textfile/0.log"
Jul 24 23:43:09.944486 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.944416    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p29hm_f12447d0-4fd8-49df-b6f5-2ac3894cb31f/kube-rbac-proxy-main/0.log"
Jul 24 23:43:09.970903 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.970870    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p29hm_f12447d0-4fd8-49df-b6f5-2ac3894cb31f/kube-rbac-proxy-self/0.log"
Jul 24 23:43:09.996756 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:09.996726    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-p29hm_f12447d0-4fd8-49df-b6f5-2ac3894cb31f/openshift-state-metrics/0.log"
Jul 24 23:43:10.296257 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:10.296185    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-hm7jj_a9859ad3-0bc1-4df1-9da3-361e6b614f23/prometheus-operator-admission-webhook/0.log"
Jul 24 23:43:11.690219 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.690107    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4fl6d_bd52e67e-d081-4e6f-93b8-5579f43a4537/networking-console-plugin/0.log"
Jul 24 23:43:11.785143 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.785090    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"]
Jul 24 23:43:11.793107 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.793073    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:11.802343 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.802318    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"]
Jul 24 23:43:11.925231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.925194    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-podres\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:11.925231 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.925241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-sys\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:11.925491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.925363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-proc\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:11.925491 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.925419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k755\" (UniqueName: \"kubernetes.io/projected/5c1d1795-fa25-4a59-865c-316af878cd82-kube-api-access-4k755\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:11.925581 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:11.925505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-lib-modules\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.026565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-lib-modules\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.026668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-podres\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.026704    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-sys\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.026765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-proc\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.026814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k755\" (UniqueName: \"kubernetes.io/projected/5c1d1795-fa25-4a59-865c-316af878cd82-kube-api-access-4k755\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.027354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-lib-modules\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.027456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-podres\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.027506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-sys\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.027634 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.027551    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c1d1795-fa25-4a59-865c-316af878cd82-proc\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.036082 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.036051    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k755\" (UniqueName: \"kubernetes.io/projected/5c1d1795-fa25-4a59-865c-316af878cd82-kube-api-access-4k755\") pod \"perf-node-gather-daemonset-n64gr\" (UID: \"5c1d1795-fa25-4a59-865c-316af878cd82\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.110441 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.110400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.352101 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.350971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"]
Jul 24 23:43:12.352827 ip-10-0-143-78 kubenswrapper[2564]: W0724 23:43:12.352762    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5c1d1795_fa25_4a59_865c_316af878cd82.slice/crio-f57a79b6fc63668ffa516412c08db7dc1f55c9a21fb9ee05710776965ab3b947 WatchSource:0}: Error finding container f57a79b6fc63668ffa516412c08db7dc1f55c9a21fb9ee05710776965ab3b947: Status 404 returned error can't find the container with id f57a79b6fc63668ffa516412c08db7dc1f55c9a21fb9ee05710776965ab3b947
Jul 24 23:43:12.421865 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.421834    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/2.log"
Jul 24 23:43:12.428212 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.428177    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-8qwwz_232174b4-8fff-4aeb-8d4b-13a5dd3f4bb3/console-operator/3.log"
Jul 24 23:43:12.547924 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.547423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr" event={"ID":"5c1d1795-fa25-4a59-865c-316af878cd82","Type":"ContainerStarted","Data":"fd52d09efe79365c2382b7548487cb7b5c4378f59898ce28c9b2d88be93ee6c5"}
Jul 24 23:43:12.547924 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.547509    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr" event={"ID":"5c1d1795-fa25-4a59-865c-316af878cd82","Type":"ContainerStarted","Data":"f57a79b6fc63668ffa516412c08db7dc1f55c9a21fb9ee05710776965ab3b947"}
Jul 24 23:43:12.548206 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.547995    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:12.580155 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:12.580064    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr" podStartSLOduration=1.580042518 podStartE2EDuration="1.580042518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:43:12.578474883 +0000 UTC m=+3053.431801423" watchObservedRunningTime="2026-07-24 23:43:12.580042518 +0000 UTC m=+3053.433369059"
Jul 24 23:43:13.048707 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:13.048668    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5658d96d79-vw77c_a8d1f2a5-3a0e-4d63-9df6-da62a6ae675a/console/0.log"
Jul 24 23:43:13.086488 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:13.086460    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-2sngs_ac9b9a00-cb92-46cb-b57f-fd50c71e1d46/download-server/0.log"
Jul 24 23:43:14.723135 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:14.723093    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pqdb6_a4e78f7d-9f01-4fa3-8913-246be64bd119/dns/0.log"
Jul 24 23:43:14.743527 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:14.743500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pqdb6_a4e78f7d-9f01-4fa3-8913-246be64bd119/kube-rbac-proxy/0.log"
Jul 24 23:43:14.813559 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:14.813525    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9vcd8_02648613-32e6-4cc7-aaab-e39b8876124d/dns-node-resolver/0.log"
Jul 24 23:43:15.464954 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:15.464920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cn5mp_011616e0-ac21-4ad2-979f-baaed2fd1174/node-ca/0.log"
Jul 24 23:43:16.418605 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:16.418577    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft_2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad/istio-proxy/0.log"
Jul 24 23:43:16.729104 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:16.729027    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-bt5q9_d22bdb04-157a-4ccd-bab8-c672428aada2/kube-auth-proxy/0.log"
Jul 24 23:43:16.850203 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:16.850175    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kc9w5_b244bf57-7ebf-4e84-8661-5ce003e1bfe3/payload-pre-processing/0.log"
Jul 24 23:43:17.786340 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:17.786313    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rpbq7_30e3c265-ecf4-40d5-910e-cde6902ec81e/serve-healthcheck-canary/0.log"
Jul 24 23:43:18.351834 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:18.351802    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fps4r_9ad14177-7c17-4970-97e9-f22e67ee5712/kube-rbac-proxy/0.log"
Jul 24 23:43:18.372533 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:18.372502    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fps4r_9ad14177-7c17-4970-97e9-f22e67ee5712/exporter/0.log"
Jul 24 23:43:18.392583 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:18.392547    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fps4r_9ad14177-7c17-4970-97e9-f22e67ee5712/extractor/0.log"
Jul 24 23:43:18.573660 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:18.573621    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-n64gr"
Jul 24 23:43:20.977429 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:20.977398    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jr2bk_bc715d7e-142f-4c70-a2c9-86cbd9077540/manager/0.log"
Jul 24 23:43:21.221569 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:21.221485    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r4q4f_95265a02-5c3d-4466-90cb-d295ef074376/manager/0.log"
Jul 24 23:43:21.480039 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:21.479957    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z79ww_3185626a-2701-4b83-9807-58def62c683f/server/0.log"
Jul 24 23:43:21.572235 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:21.572208    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vthzr_a36ff974-d2d8-4de7-ac17-b7885415da66/manager/0.log"
Jul 24 23:43:21.828744 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:21.828656    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-qztx5_f26aa5f2-483b-449d-a43a-c018b9c75ebc/manager/0.log"
Jul 24 23:43:21.851284 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:21.851257    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-qztx5_f26aa5f2-483b-449d-a43a-c018b9c75ebc/copy-manifests/0.log"
Jul 24 23:43:22.891467 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:22.891380    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft_2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad/istio-proxy/0.log"
Jul 24 23:43:23.180888 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:23.180796    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-bt5q9_d22bdb04-157a-4ccd-bab8-c672428aada2/kube-auth-proxy/0.log"
Jul 24 23:43:23.291475 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:23.291443    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kc9w5_b244bf57-7ebf-4e84-8661-5ce003e1bfe3/payload-pre-processing/0.log"
Jul 24 23:43:24.700772 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:24.700739    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-g7gft_2d9c4c56-d1b4-40ff-8b5f-e8f6232520ad/istio-proxy/0.log"
Jul 24 23:43:24.998393 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:24.998323    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-bt5q9_d22bdb04-157a-4ccd-bab8-c672428aada2/kube-auth-proxy/0.log"
Jul 24 23:43:25.109838 ip-10-0-143-78 kubenswrapper[2564]: I0724 23:43:25.109806    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kc9w5_b244bf57-7ebf-4e84-8661-5ce003e1bfe3/payload-pre-processing/0.log"