Jul 07 02:07:39.621212 ip-10-0-143-152 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 02:07:39.621221 ip-10-0-143-152 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 02:07:39.621228 ip-10-0-143-152 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 02:07:39.621450 ip-10-0-143-152 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 02:07:49.734929 ip-10-0-143-152 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 02:07:49.734952 ip-10-0-143-152 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 760167d8cb114395996d300e4f0abaa5 --
Jul 07 02:09:48.540272 ip-10-0-143-152 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.090980    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094631    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094647    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094652    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094656    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094659    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094663    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094666    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:09:50.108098 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094668    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094671    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094673    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094676    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094679    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094681    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094684    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094686    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094689    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094691    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094694    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094696    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094699    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094701    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094704    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094707    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094709    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094712    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094717    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094721    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:09:50.109207 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094724    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094727    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094730    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094732    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094735    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094738    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094740    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094743    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094746    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094749    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094751    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094754    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094757    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094759    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094762    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094765    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094767    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094771    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094774    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094777    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:09:50.110288 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094781    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094784    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094787    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094789    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094792    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094795    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094797    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094800    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094802    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094805    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094808    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094810    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094813    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094816    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094819    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094821    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094824    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094826    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094829    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094832    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:09:50.111152 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094835    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094837    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094840    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094843    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094846    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094848    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094852    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094855    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094857    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094860    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094862    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094865    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094868    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094870    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094872    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094877    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094880    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094883    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094886    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:09:50.112508 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094889    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094892    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094894    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094897    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094899    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094902    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094905    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094908    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094911    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094913    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094916    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094919    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.094922    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095650    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095657    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095661    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095664    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095667    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095670    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095674    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:09:50.113380 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095676    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095679    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095682    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095684    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095687    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095690    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095692    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095695    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095697    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095700    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095703    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095707    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095712    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095715    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095718    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095721    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095724    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095726    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095729    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:09:50.115346 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095731    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095734    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095738    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095742    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095745    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095749    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095752    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095754    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095757    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095759    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095762    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095765    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095767    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095770    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095773    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095776    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095779    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095781    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095784    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095786    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:09:50.116228 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095789    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095792    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095795    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095797    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095800    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095803    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095805    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095808    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095811    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095814    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095816    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095819    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095821    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095824    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095826    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095828    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095831    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095834    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095837    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095840    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:09:50.117759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095843    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095845    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095848    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095850    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095853    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095856    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095859    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095862    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095864    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095867    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095869    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095872    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095874    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095877    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095879    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095882    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095885    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095887    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095890    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095892    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:09:50.118643 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095895    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095898    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095901    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095906    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095909    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095912    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095915    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095918    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095921    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095924    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095926    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095929    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.095932    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096330    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096335    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096338    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096341    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096344    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096346    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096349    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:09:50.119740 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096352    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096355    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096358    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096360    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096363    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096365    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096368    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096371    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096374    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096376    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096379    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096382    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096386    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096389    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096392    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096396    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096399    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096401    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096404    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096407    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:09:50.120617 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096409    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096412    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096414    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096417    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096420    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096422    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096425    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096427    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096430    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096432    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096436    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096442    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096445    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096448    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096450    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096453    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096455    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096458    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096460    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096463    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:09:50.122816 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096466    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096468    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096471    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096473    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096478    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096480    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096483    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096485    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096488    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096504    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096507    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096510    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096513    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096516    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096519    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096521    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096524    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096527    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096530    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096533    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:09:50.123911 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096535    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096538    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096542    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096547    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096551    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096554    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096557    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096560    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096562    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096565    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096568    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096570    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096573    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096576    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096578    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096581    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096585    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096589    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096591    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:09:50.125233 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096594    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096597    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096600    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096603    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096605    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096608    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096610    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096613    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096615    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096618    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096621    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096624    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.096626    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096699    2568 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096709    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096716    2568 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096722    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096729    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096732    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096737    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096741    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 02:09:50.126095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096745    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096748    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096752    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096755    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096758    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096761    2568 flags.go:64] FLAG: --cgroup-root=""
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096764    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096767    2568 flags.go:64] FLAG: --client-ca-file=""
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096772    2568 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096775    2568 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096779    2568 flags.go:64] FLAG: --cluster-domain=""
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096782    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096785    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096788    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096791    2568 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096795    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096798    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096801    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096804    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096807    2568 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096810    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096813    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096817    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096820    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 02:09:50.134122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096825    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096827    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096830    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096835    2568 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096838    2568 flags.go:64] FLAG: --enable-server="true"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096841    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096847    2568 flags.go:64] FLAG: --event-burst="100"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096850    2568 flags.go:64] FLAG: --event-qps="50"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096853    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096856    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096859    2568 flags.go:64] FLAG: --eviction-hard=""
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096863    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096866    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096869    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096872    2568 flags.go:64] FLAG: --eviction-soft=""
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096875    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096879    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096882    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096885    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096888    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096891    2568 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096894    2568 flags.go:64] FLAG: --feature-gates=""
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096897    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096900    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096903    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 02:09:50.135156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096906    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096909    2568 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096912    2568 flags.go:64] FLAG: --help="false"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096915    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096919    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096922    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096925    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096928    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096932    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096935    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096939    2568 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096942    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096945    2568 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096948    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096951    2568 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096953    2568 flags.go:64] FLAG: --kube-reserved=""
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096956    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096959    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096962    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096965    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096967    2568 flags.go:64] FLAG: --lock-file=""
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096970    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096973    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096977    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 02:09:50.137131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096983    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096985    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096988    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096991    2568 flags.go:64] FLAG: --logging-format="text"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096994    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.096997    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097000    2568 flags.go:64] FLAG: --manifest-url=""
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097003    2568 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097007    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097009    2568 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097014    2568 flags.go:64] FLAG: --max-pods="110"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097016    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097020    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097022    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097025    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097029    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097032    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097035    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097043    2568 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097046    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097049    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097053    2568 flags.go:64] FLAG: --pod-cidr=""
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097056    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 02:09:50.138253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097061    2568 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097064    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097067    2568 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097070    2568 flags.go:64] FLAG: --port="10250"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097073    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097076    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03decb137598dd981"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097079    2568 flags.go:64] FLAG: --qos-reserved=""
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097082    2568 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097085    2568 flags.go:64] FLAG: --register-node="true"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097090    2568 flags.go:64] FLAG: --register-with-taints=""
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097093    2568 flags.go:64] FLAG: --registry-burst="10"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097096    2568 flags.go:64] FLAG: --registry-qps="5"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097099    2568 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097114    2568 flags.go:64] FLAG: --reserved-memory=""
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097117    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097121    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097124    2568 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097127    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097130    2568 flags.go:64] FLAG: --runonce="false"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097133    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097136    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097139    2568 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097142    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097144    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097147    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097152    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 02:09:50.140268 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097155    2568 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097159    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097162    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097165    2568 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097168    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097171    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097174    2568 flags.go:64] FLAG: --system-cgroups=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097177    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097183    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097185    2568 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097189    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097194    2568 flags.go:64] FLAG: --tls-min-version=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097197    2568 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097207    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097210    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097215    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097233    2568 flags.go:64] FLAG: --v="2"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097238    2568 flags.go:64] FLAG: --version="false"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097243    2568 flags.go:64] FLAG: --vmodule=""
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097247    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.097250    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097381    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097385    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097388    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:09:50.141370 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097391    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097394    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097397    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097400    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097403    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097406    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097409    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097413    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097420    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097426    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097428    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097431    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097434    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097436    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097439    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097441    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097444    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097447    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097450    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097452    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:09:50.226449 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097455    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:09:50.196610 ip-10-0-143-152 systemd[1]: Started Kubernetes Kubelet.
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097458    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097460    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097464    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097467    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097469    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097472    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097474    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097477    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097479    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097482    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097484    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097487    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097503    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097506    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097509    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097511    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097514    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097517    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097519    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:09:50.227392 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097522    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097526    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097529    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097532    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097535    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097537    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097540    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097542    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097545    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097547    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097550    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097552    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097555    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097557    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097560    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097563    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097567    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097569    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097572    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097575    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:09:50.228851 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097577    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097580    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097582    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097586    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097589    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097592    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097595    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097597    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097600    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097602    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097605    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097607    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097611    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097616    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097619    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097622    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097624    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097627    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:09:50.229555 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097630    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097633    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097635    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097638    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097641    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097644    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097647    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097650    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097652    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097656    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097658    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097661    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097664    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097666    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097669    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097671    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097674    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:09:50.230543 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.097676    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.098353    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.107566    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.107584    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.108611    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.108619    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108689    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108697    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108701    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108704    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108708    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108712    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108716    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108720    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:09:50.231436 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108723    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108726    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108729    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108732    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108735    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108737    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108740    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108742    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108746    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108748    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108751    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108755    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108759    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108761    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108764    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108769    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108771    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108774    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108778    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108782    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:09:50.232552 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108785    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108794    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108798    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108802    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108806    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108809    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108811    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108815    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108818    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108821    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108824    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108827    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108830    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108833    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108836    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108839    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108842    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108844    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108847    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:09:50.233192 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108851    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108854    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108856    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108859    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108863    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108865    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108868    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108872    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108875    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108878    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108883    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108888    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108891    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108894    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108898    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108909    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108913    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108915    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108918    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108922    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:09:50.236045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108925    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108928    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108931    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108934    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108937    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108940    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108943    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108946    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108950    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108952    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108955    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108958    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108960    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108963    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108965    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108968    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108971    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108973    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108977    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108980    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:09:50.236893 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108982    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108985    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108987    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108991    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108994    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108997    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.108999    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109003    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109012    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109017    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109019    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109024    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.109030    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109197    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109211    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:09:50.238900 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109214    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109217    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109220    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109224    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109227    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109229    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109233    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109237    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109240    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109242    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109246    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109249    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109251    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109254    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109260    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109263    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109265    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109268    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109273    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109275    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:09:50.239523 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109278    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109282    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109284    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109287    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109290    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109293    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109303    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109307    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109311    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109315    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109318    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109321    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109323    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109327    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109330    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109332    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109335    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109338    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109342    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109344    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:09:50.240293 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109347    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109350    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109354    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109357    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109360    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109363    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109367    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109370    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109372    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109376    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109379    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109381    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109385    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109389    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109391    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109394    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109396    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109399    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109401    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:09:50.240880 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109411    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109416    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109419    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109422    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109424    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109428    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109431    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109434    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109436    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109439    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109442    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109445    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109448    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109451    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109454    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109457    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109460    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109464    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109469    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:09:50.342983 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109473    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109478    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109481    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109483    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109487    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109505    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109509    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109514    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109516    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109519    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109522    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109526    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109528    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109531    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109543    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109546    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109551    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109554    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:09:50.359048 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:50.109558    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.109563    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.109707    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.111943    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.113629    2568 server.go:1019] "Starting client certificate rotation"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.116330    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.117306    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.117359    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.152753    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.161842    2568 log.go:25] "Validated CRI v1 runtime API"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.164028    2568 log.go:25] "Validated CRI v1 image API"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.164921    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 02:09:50.362054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.169131    2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 907f548f-86e0-4a73-a5db-f68a3cd404d2:/dev/nvme0n1p3 a7d9abb3-12f7-49c9-913a-deb2b773df44:/dev/nvme0n1p4]
Jul 07 02:09:50.362910 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.169150    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.174285    2568 manager.go:216] Machine: {Timestamp:2026-07-07 02:09:50.172944335 +0000 UTC m=+0.376365069 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092597 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd3d6708c53f71d028587ae363aa5 SystemUUID:ec2cd3d6-708c-53f7-1d02-8587ae363aa5 BootID:760167d8-cb11-4395-996d-300e4f0abaa5 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d4:63:30:50:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d4:63:30:50:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:63:12:38:bb:c2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.174388    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.174470    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.177296    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.177322    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-152.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.178668    2568 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.178682    2568 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 02:09:50.364816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.178690    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.178701    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.180020    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.181774    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183197    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183303    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183332    2568 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183341    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183354    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183362    2568 kubelet.go:398] "Adding apiserver pod source"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183370    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183710    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.183729    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.186636    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.188517    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.188545    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190182    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190194    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190200    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190206    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190214    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190220    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190227    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190233    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190238    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190243    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190256    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.190262    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.191610    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.191618    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.195566    2568 server.go:1268] "Started kubelet"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.195603    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.195721    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.195788    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.197173    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.197932    2568 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.198660    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-152.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.198868    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.198905    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 02:09:50.365475 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.200438    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.204826    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.204856    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.205532    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.205836    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.205853    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.205871    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.205965    2568 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.205971    2568 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210221    2568 factory.go:55] Registering systemd factory
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210242    2568 factory.go:223] Registration of the systemd container factory successfully
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210480    2568 factory.go:153] Registering CRI-O factory
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210512    2568 factory.go:223] Registration of the crio container factory successfully
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210563    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210592    2568 factory.go:103] Registering Raw factory
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.210610    2568 manager.go:1196] Started watching for new ooms in manager
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.211340    2568 manager.go:349] Starting recovery of all containers
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.213869    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.213989    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.213818    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-152.ec2.internal.18bfdfedde84a2d9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-152.ec2.internal,UID:ip-10-0-143-152.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-152.ec2.internal,},FirstTimestamp:2026-07-07 02:09:50.195483353 +0000 UTC m=+0.398904088,LastTimestamp:2026-07-07 02:09:50.195483353 +0000 UTC m=+0.398904088,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-152.ec2.internal,}"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.227563    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjtnc"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.230630    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjtnc"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.231204    2568 manager.go:354] Recovery completed
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.240584    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.242462    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.242488    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.242514    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243045    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243053    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243055    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243095    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243679    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.368501 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243702    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.243717    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.244131    2568 policy_none.go:49] "None policy: Start"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.244144    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.244155    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.244820    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-152.ec2.internal.18bfdfede151aa34  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-152.ec2.internal,UID:ip-10-0-143-152.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-152.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-152.ec2.internal,},FirstTimestamp:2026-07-07 02:09:50.242474548 +0000 UTC m=+0.445895286,LastTimestamp:2026-07-07 02:09:50.242474548 +0000 UTC m=+0.445895286,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-152.ec2.internal,}"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.245097    2568 policy_none.go:47] "Start"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.306057    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311125    2568 manager.go:337] "Starting Device Plugin manager"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.311154    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311163    2568 server.go:87] "Starting device plugin registration server"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311435    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311451    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311543    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311629    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.311637    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.312191    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 02:09:50.370034 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.312240    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.412017    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.412993    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.413020    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.413030    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.413057    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.419690    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.419718    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-152.ec2.internal\": node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.425371    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.426625    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.426665    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.426684    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.426712    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.429268    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:09:50.498951 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.436736    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.527184    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"]
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.527268    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.530672    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.530700    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.530710    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.531894    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.532001    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.532030    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.532955    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.532987    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.533005    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.533019    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.532988    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.533071    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.534162    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.534183    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.534845    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.534874    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.534891    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.537551    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.565107    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-152.ec2.internal\" not found" node="ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.579013 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.570014    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-152.ec2.internal\" not found" node="ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.607643 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.607618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fbaaf62e506e0ad292b5345120621ade-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal\" (UID: \"fbaaf62e506e0ad292b5345120621ade\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.607643 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.607644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbaaf62e506e0ad292b5345120621ade-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal\" (UID: \"fbaaf62e506e0ad292b5345120621ade\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.607784 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.607663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5a6799705e4948b663271414d9e6b73-config\") pod \"kube-apiserver-proxy-ip-10-0-143-152.ec2.internal\" (UID: \"a5a6799705e4948b663271414d9e6b73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.638044 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.638020    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.707961 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.707931    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fbaaf62e506e0ad292b5345120621ade-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal\" (UID: \"fbaaf62e506e0ad292b5345120621ade\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.707961 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.707963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbaaf62e506e0ad292b5345120621ade-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal\" (UID: \"fbaaf62e506e0ad292b5345120621ade\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.708165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.707982    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5a6799705e4948b663271414d9e6b73-config\") pod \"kube-apiserver-proxy-ip-10-0-143-152.ec2.internal\" (UID: \"a5a6799705e4948b663271414d9e6b73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.708165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.708030    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbaaf62e506e0ad292b5345120621ade-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal\" (UID: \"fbaaf62e506e0ad292b5345120621ade\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.708165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.708033    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fbaaf62e506e0ad292b5345120621ade-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal\" (UID: \"fbaaf62e506e0ad292b5345120621ade\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.708165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.708097    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5a6799705e4948b663271414d9e6b73-config\") pod \"kube-apiserver-proxy-ip-10-0-143-152.ec2.internal\" (UID: \"a5a6799705e4948b663271414d9e6b73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.739080 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.739034    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.839624 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.839535    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:50.867670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.867639    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.872917 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:50.872889    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:50.939733 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:50.939696    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:51.040282 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:51.040250    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:51.112886 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.112787    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 02:09:51.113520 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.112977    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 02:09:51.140973 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:51.140945    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:51.191272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.191242    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:09:51.234178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.234130    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 02:04:50 +0000 UTC" deadline="2027-12-08 06:55:24.88307807 +0000 UTC"
Jul 07 02:09:51.234178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.234169    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12460h45m33.648912588s"
Jul 07 02:09:51.241274 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:51.241256    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:51.341959 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:51.341919    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:51.442218 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:51.442161    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-152.ec2.internal\" not found"
Jul 07 02:09:51.456172 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.456152    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:09:51.506427 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.506398    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:51.525809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.525785    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 02:09:51.528657 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.528642    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal"
Jul 07 02:09:51.540318 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.540299    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 02:09:51.564371 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:51.564341    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbaaf62e506e0ad292b5345120621ade.slice/crio-719973065c0f8753370c1fc3ae68a54e6014886e0c5e67d864d6ec6bd136e84a WatchSource:0}: Error finding container 719973065c0f8753370c1fc3ae68a54e6014886e0c5e67d864d6ec6bd136e84a: Status 404 returned error can't find the container with id 719973065c0f8753370c1fc3ae68a54e6014886e0c5e67d864d6ec6bd136e84a
Jul 07 02:09:51.564565 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:51.564550    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a6799705e4948b663271414d9e6b73.slice/crio-9b3af6d0c450282701be67cfc6d6f31697554c219ad3af271a96411fb12cc3ea WatchSource:0}: Error finding container 9b3af6d0c450282701be67cfc6d6f31697554c219ad3af271a96411fb12cc3ea: Status 404 returned error can't find the container with id 9b3af6d0c450282701be67cfc6d6f31697554c219ad3af271a96411fb12cc3ea
Jul 07 02:09:51.569484 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.569471    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:09:51.707713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:51.707630    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:09:52.107477 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.107398    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:09:52.184557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.184522    2568 apiserver.go:52] "Watching apiserver"
Jul 07 02:09:52.193783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.193745    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 02:09:52.195140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.195111    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j","openshift-cluster-node-tuning-operator/tuned-pd5z6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal","openshift-multus/multus-additional-cni-plugins-xrrzv","openshift-multus/network-metrics-daemon-mzcc5","openshift-network-diagnostics/network-check-target-slv96","kube-system/global-pull-secret-syncer-q9bb5","openshift-dns/node-resolver-l2wnp","openshift-image-registry/node-ca-r7s9h","openshift-multus/multus-mrpsn","openshift-network-operator/iptables-alerter-z9grv","openshift-ovn-kubernetes/ovnkube-node-z9t55","kube-system/konnectivity-agent-57vj2"]
Jul 07 02:09:52.197604 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.196906    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.201568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.200646    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.202863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.202343    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.202863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.202427    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 02:09:52.202863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.202599    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.202863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.202687    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tw4wg\""
Jul 07 02:09:52.203103 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.202917    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v2zzd\""
Jul 07 02:09:52.203162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.203113    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.205092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.203469    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 02:09:52.205092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.203698    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 02:09:52.205449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.205317    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 02:09:52.205720 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.205698    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 02:09:52.208749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.205833    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.208749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.206023    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 02:09:52.210910 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.209414    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.212433 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.211570    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.212433 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.211683    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.212433 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.211734    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tn7hn\""
Jul 07 02:09:52.213161 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.212844    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.214663 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214599    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovnkube-script-lib\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.214663 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysctl-conf\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214664    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-sys\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrtwj\" (UniqueName: \"kubernetes.io/projected/d500a857-2643-4108-80af-8da289f0f3c7-kube-api-access-jrtwj\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214715    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovn-node-metrics-cert\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214738    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986445b3-5f4c-4975-9057-389f636b8f79-tmp\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214759    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d500a857-2643-4108-80af-8da289f0f3c7-host\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214780    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-cni-netd\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.214811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214807    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysctl-d\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214829    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-var-lib-kubelet\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214851    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-host\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214873    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d500a857-2643-4108-80af-8da289f0f3c7-serviceca\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214896    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-systemd-units\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214919    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214941    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-ovn\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214964    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-kubelet\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.214990    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-run-netns\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-modprobe-d\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215049    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysconfig\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-systemd\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215101    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-run\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215122    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/986445b3-5f4c-4975-9057-389f636b8f79-etc-tuned\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215145    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-log-socket\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215167    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215189    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-cni-bin\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215213    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-kubernetes\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215237    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qth\" (UniqueName: \"kubernetes.io/projected/986445b3-5f4c-4975-9057-389f636b8f79-kube-api-access-s9qth\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215266    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-systemd\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-var-lib-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215314    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-env-overrides\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215376    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9l9d\" (UniqueName: \"kubernetes.io/projected/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-kube-api-access-g9l9d\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215402    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-slash\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215425    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-etc-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215450    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-node-log\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.215876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215484    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-lib-modules\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.216631 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.215521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovnkube-config\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.216631 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.216205    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 02:09:52.216631 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.216288    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 02:09:52.216631 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.216582    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tl5d4\""
Jul 07 02:09:52.217581 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.216979    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.219034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.219014    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 02:09:52.219296 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.219275    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-76nzr\""
Jul 07 02:09:52.220387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.219694    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.220387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.219854    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 02:09:52.220387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.219980    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.220387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.220131    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 02:09:52.220387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.220135    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.220387 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.220200    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:09:52.220760 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.220424    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 02:09:52.223130 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.223112    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.224660 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.224643    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:52.224890 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.224867    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:09:52.226179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.225591    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.226179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.225945    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.226179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.226131    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cj662\""
Jul 07 02:09:52.231068 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.230381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.232377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.231759    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.232377 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.231860    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:09:52.233063 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.233045    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.233216 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.233108    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kkhjl\""
Jul 07 02:09:52.233309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.233293    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.233382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.233299    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 02:09:52.233446 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.233435    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.236291 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.236202    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xbm5"
Jul 07 02:09:52.236291 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.236208    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-phwsk\""
Jul 07 02:09:52.236291 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.236256    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 02:09:52.236671 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.236581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.238665 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.238645    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:09:52.239172 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.238949    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 02:09:52.239172 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.238983    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f5j5d\""
Jul 07 02:09:52.239359 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.239300    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 02:09:52.243570 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.243552    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xbm5"
Jul 07 02:09:52.307267 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.307239    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 02:09:52.316231 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-conf-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.316231 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316222    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986445b3-5f4c-4975-9057-389f636b8f79-tmp\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.316231 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316245    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-cni-netd\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.316231 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316273    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-cni-multus\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316333    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-cni-netd\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5eedf342-ae5c-45ce-bc64-474b56286922-multus-daemon-config\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-device-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316416    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4358822-184a-4fa7-8d72-27b1af7c6d75-dbus\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316440    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-var-lib-kubelet\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316482    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d500a857-2643-4108-80af-8da289f0f3c7-serviceca\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316524    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-var-lib-kubelet\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-systemd-units\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqcwk\" (UniqueName: \"kubernetes.io/projected/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-kube-api-access-qqcwk\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-os-release\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.316655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316644    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-systemd-units\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5eedf342-ae5c-45ce-bc64-474b56286922-cni-binary-copy\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316698    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-multus-certs\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316723    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de44aa3f-5598-4425-9d60-5b70370a9493-iptables-alerter-script\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316733    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316747    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316775    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-k8s-cni-cncf-io\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316824    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-socket-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316849    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bec11113-359b-4151-bd2b-cf5937c9879e-agent-certs\") pod \"konnectivity-agent-57vj2\" (UID: \"bec11113-359b-4151-bd2b-cf5937c9879e\") " pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316875    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-modprobe-d\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316930    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-run\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316953    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/986445b3-5f4c-4975-9057-389f636b8f79-etc-tuned\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.316976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-log-socket\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317016    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d500a857-2643-4108-80af-8da289f0f3c7-serviceca\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317024    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-cni-bin\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317059    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cnibin\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317106    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-kubernetes\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317128    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-systemd\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317152    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-var-lib-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317181    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-env-overrides\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-socket-dir-parent\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317255    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-etc-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317280    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-netns\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317298    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-run\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-etc-kubernetes\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317326    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-lib-modules\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.317953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317421    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-log-socket\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317421    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-cni-bin\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317476    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovnkube-script-lib\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317484    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-kubernetes\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317527    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysctl-conf\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317606    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-etc-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317635    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-modprobe-d\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317738    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-lib-modules\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317738    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-sys\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317776    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-var-lib-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317794    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovn-node-metrics-cert\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317840    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-sys\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-os-release\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.318644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317909    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-kubelet\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5xg8\" (UniqueName: \"kubernetes.io/projected/5eedf342-ae5c-45ce-bc64-474b56286922-kube-api-access-k5xg8\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317955    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-env-overrides\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317978    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de44aa3f-5598-4425-9d60-5b70370a9493-host-slash\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318019    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lpdb\" (UniqueName: \"kubernetes.io/projected/de44aa3f-5598-4425-9d60-5b70370a9493-kube-api-access-2lpdb\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d500a857-2643-4108-80af-8da289f0f3c7-host\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318097    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318136    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z48vk\" (UniqueName: \"kubernetes.io/projected/b6144ddf-5525-45aa-afb7-ef69d27cab03-kube-api-access-z48vk\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.317684    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-systemd\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318160    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-hostroot\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318190    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-registration-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318228    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d500a857-2643-4108-80af-8da289f0f3c7-host\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318238    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-sys-fs\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318247    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysctl-conf\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.319272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318303    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysctl-d\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318361    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-host\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318473    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysctl-d\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318532    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318558    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovnkube-script-lib\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318565    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-ovn\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318623    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-ovn\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318642    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-cnibin\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318668    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-host\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318671    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-kubelet\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-run-netns\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318716    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-run-openvswitch\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318726    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318781    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-kubelet\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.319931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318781    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-run-netns\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318813    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-cni-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318877    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwzrn\" (UniqueName: \"kubernetes.io/projected/81581446-e6c0-4493-8636-61ae23ba83e2-kube-api-access-nwzrn\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318919    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.318953    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysconfig\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319000    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-sysconfig\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319039    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-systemd\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319075    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319119    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/986445b3-5f4c-4975-9057-389f636b8f79-etc-systemd\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319161    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319192    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4358822-184a-4fa7-8d72-27b1af7c6d75-kubelet-config\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319226    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6144ddf-5525-45aa-afb7-ef69d27cab03-hosts-file\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319251    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6144ddf-5525-45aa-afb7-ef69d27cab03-tmp-dir\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-system-cni-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.320713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319318    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qth\" (UniqueName: \"kubernetes.io/projected/986445b3-5f4c-4975-9057-389f636b8f79-kube-api-access-s9qth\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319408    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9l9d\" (UniqueName: \"kubernetes.io/projected/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-kube-api-access-g9l9d\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319475    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-cni-bin\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-slash\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319546    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-node-log\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319585    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319635    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-host-slash\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319766    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d566f\" (UniqueName: \"kubernetes.io/projected/478e76b1-c8b0-466d-aa7e-5dc477d27b41-kube-api-access-d566f\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319811    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovnkube-config\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319835    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bec11113-359b-4151-bd2b-cf5937c9879e-konnectivity-ca\") pod \"konnectivity-agent-57vj2\" (UID: \"bec11113-359b-4151-bd2b-cf5937c9879e\") " pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319883    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-node-log\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319918    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.321377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.319949    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrtwj\" (UniqueName: \"kubernetes.io/projected/d500a857-2643-4108-80af-8da289f0f3c7-kube-api-access-jrtwj\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.322193 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.320039    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986445b3-5f4c-4975-9057-389f636b8f79-tmp\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.322193 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.320581    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovnkube-config\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.322193 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.320595    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-ovn-node-metrics-cert\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.322193 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.320637    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/986445b3-5f4c-4975-9057-389f636b8f79-etc-tuned\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.327679 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.327653    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qth\" (UniqueName: \"kubernetes.io/projected/986445b3-5f4c-4975-9057-389f636b8f79-kube-api-access-s9qth\") pod \"tuned-pd5z6\" (UID: \"986445b3-5f4c-4975-9057-389f636b8f79\") " pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.327808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.327675    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9l9d\" (UniqueName: \"kubernetes.io/projected/ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e-kube-api-access-g9l9d\") pod \"ovnkube-node-z9t55\" (UID: \"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.327808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.327697    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrtwj\" (UniqueName: \"kubernetes.io/projected/d500a857-2643-4108-80af-8da289f0f3c7-kube-api-access-jrtwj\") pod \"node-ca-r7s9h\" (UID: \"d500a857-2643-4108-80af-8da289f0f3c7\") " pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.420811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420721    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.420811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420775    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-os-release\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.420811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420802    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-kubelet\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420826    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5xg8\" (UniqueName: \"kubernetes.io/projected/5eedf342-ae5c-45ce-bc64-474b56286922-kube-api-access-k5xg8\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420848    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de44aa3f-5598-4425-9d60-5b70370a9493-host-slash\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420872    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lpdb\" (UniqueName: \"kubernetes.io/projected/de44aa3f-5598-4425-9d60-5b70370a9493-kube-api-access-2lpdb\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420897    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420908    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420922    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z48vk\" (UniqueName: \"kubernetes.io/projected/b6144ddf-5525-45aa-afb7-ef69d27cab03-kube-api-access-z48vk\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.420979    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-hostroot\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421005    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-registration-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-sys-fs\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.421092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421089    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-cnibin\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421116    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421128    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-hostroot\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421146    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-cni-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421157    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-sys-fs\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421171    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwzrn\" (UniqueName: \"kubernetes.io/projected/81581446-e6c0-4493-8636-61ae23ba83e2-kube-api-access-nwzrn\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421175    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-os-release\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421228    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-cnibin\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421258    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-kubelet\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421261    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421269    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de44aa3f-5598-4425-9d60-5b70370a9493-host-slash\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421325    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-registration-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421436    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421468    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-cni-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.421624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421506    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-system-cni-dir\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421488    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4358822-184a-4fa7-8d72-27b1af7c6d75-kubelet-config\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421563    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6144ddf-5525-45aa-afb7-ef69d27cab03-hosts-file\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421579    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6144ddf-5525-45aa-afb7-ef69d27cab03-tmp-dir\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-system-cni-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421639    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-cni-bin\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421648    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4358822-184a-4fa7-8d72-27b1af7c6d75-kubelet-config\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421658    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d566f\" (UniqueName: \"kubernetes.io/projected/478e76b1-c8b0-466d-aa7e-5dc477d27b41-kube-api-access-d566f\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421749    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6144ddf-5525-45aa-afb7-ef69d27cab03-hosts-file\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421843    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-system-cni-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421856    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421862    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421868    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-cni-bin\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.422218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bec11113-359b-4151-bd2b-cf5937c9879e-konnectivity-ca\") pod \"konnectivity-agent-57vj2\" (UID: \"bec11113-359b-4151-bd2b-cf5937c9879e\") " pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421935    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421962    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-conf-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.421988    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-cni-multus\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422052    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5eedf342-ae5c-45ce-bc64-474b56286922-multus-daemon-config\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422056    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-conf-dir\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422102    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6144ddf-5525-45aa-afb7-ef69d27cab03-tmp-dir\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422109    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-var-lib-cni-multus\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.422116    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422140    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-device-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422165    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4358822-184a-4fa7-8d72-27b1af7c6d75-dbus\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.422222    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:52.922175079 +0000 UTC m=+3.125595821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422258    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqcwk\" (UniqueName: \"kubernetes.io/projected/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-kube-api-access-qqcwk\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422285    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4358822-184a-4fa7-8d72-27b1af7c6d75-dbus\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.423012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-os-release\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5eedf342-ae5c-45ce-bc64-474b56286922-cni-binary-copy\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422341    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-multus-certs\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422369    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bec11113-359b-4151-bd2b-cf5937c9879e-konnectivity-ca\") pod \"konnectivity-agent-57vj2\" (UID: \"bec11113-359b-4151-bd2b-cf5937c9879e\") " pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422368    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de44aa3f-5598-4425-9d60-5b70370a9493-iptables-alerter-script\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-k8s-cni-cncf-io\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422517    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-socket-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bec11113-359b-4151-bd2b-cf5937c9879e-agent-certs\") pod \"konnectivity-agent-57vj2\" (UID: \"bec11113-359b-4151-bd2b-cf5937c9879e\") " pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422574    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422598    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cnibin\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-socket-dir-parent\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422653    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-netns\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.423664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422676    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-etc-kubernetes\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422755    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-etc-kubernetes\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422901    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de44aa3f-5598-4425-9d60-5b70370a9493-iptables-alerter-script\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422972    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-multus-certs\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.422342    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-device-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423050    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-os-release\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423099    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5eedf342-ae5c-45ce-bc64-474b56286922-cni-binary-copy\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423163    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423176    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5eedf342-ae5c-45ce-bc64-474b56286922-multus-daemon-config\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423206    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-k8s-cni-cncf-io\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423214    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423280    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81581446-e6c0-4493-8636-61ae23ba83e2-socket-dir\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423280    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-multus-socket-dir-parent\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423280    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5eedf342-ae5c-45ce-bc64-474b56286922-host-run-netns\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.423320    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-cnibin\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.424246 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.423323    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:52.424998 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.423367    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:52.923354211 +0000 UTC m=+3.126774946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:52.425689 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.425663    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bec11113-359b-4151-bd2b-cf5937c9879e-agent-certs\") pod \"konnectivity-agent-57vj2\" (UID: \"bec11113-359b-4151-bd2b-cf5937c9879e\") " pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.427234 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.427213    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:09:52.427353 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.427239    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:09:52.427353 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.427254    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:52.427353 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.427320    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:52.927302444 +0000 UTC m=+3.130723186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:52.429735 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.429710    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwzrn\" (UniqueName: \"kubernetes.io/projected/81581446-e6c0-4493-8636-61ae23ba83e2-kube-api-access-nwzrn\") pod \"aws-ebs-csi-driver-node-6wq6j\" (UID: \"81581446-e6c0-4493-8636-61ae23ba83e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.431510 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.431328    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lpdb\" (UniqueName: \"kubernetes.io/projected/de44aa3f-5598-4425-9d60-5b70370a9493-kube-api-access-2lpdb\") pod \"iptables-alerter-z9grv\" (UID: \"de44aa3f-5598-4425-9d60-5b70370a9493\") " pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.431510 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.431444    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d566f\" (UniqueName: \"kubernetes.io/projected/478e76b1-c8b0-466d-aa7e-5dc477d27b41-kube-api-access-d566f\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.432431 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.432131    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5xg8\" (UniqueName: \"kubernetes.io/projected/5eedf342-ae5c-45ce-bc64-474b56286922-kube-api-access-k5xg8\") pod \"multus-mrpsn\" (UID: \"5eedf342-ae5c-45ce-bc64-474b56286922\") " pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.432729 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.432637    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqcwk\" (UniqueName: \"kubernetes.io/projected/ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7-kube-api-access-qqcwk\") pod \"multus-additional-cni-plugins-xrrzv\" (UID: \"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7\") " pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.432729 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.432649    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z48vk\" (UniqueName: \"kubernetes.io/projected/b6144ddf-5525-45aa-afb7-ef69d27cab03-kube-api-access-z48vk\") pod \"node-resolver-l2wnp\" (UID: \"b6144ddf-5525-45aa-afb7-ef69d27cab03\") " pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.433961 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.433913    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal" event={"ID":"fbaaf62e506e0ad292b5345120621ade","Type":"ContainerStarted","Data":"719973065c0f8753370c1fc3ae68a54e6014886e0c5e67d864d6ec6bd136e84a"}
Jul 07 02:09:52.436144 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.436111    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal" event={"ID":"a5a6799705e4948b663271414d9e6b73","Type":"ContainerStarted","Data":"9b3af6d0c450282701be67cfc6d6f31697554c219ad3af271a96411fb12cc3ea"}
Jul 07 02:09:52.516575 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.516538    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r7s9h"
Jul 07 02:09:52.529043 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.529004    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:09:52.537904 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.537880    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pd5z6"
Jul 07 02:09:52.547050 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.547024    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:09:52.553790 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.553770    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrrzv"
Jul 07 02:09:52.562749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.562721    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2wnp"
Jul 07 02:09:52.569624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.569604    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j"
Jul 07 02:09:52.576364 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.576339    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mrpsn"
Jul 07 02:09:52.581946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.581928    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9grv"
Jul 07 02:09:52.927120 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.927088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:52.927297 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:52.927142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:52.927297 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.927244    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:52.927297 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.927274    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:52.927418 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.927315    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:53.927294125 +0000 UTC m=+4.130714862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:52.927418 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:52.927338    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:53.927324945 +0000 UTC m=+4.130745680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:53.028343 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.028298    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:53.028553 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.028462    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:09:53.028553 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.028489    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:09:53.028553 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.028517    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:53.028728 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.028587    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:54.028567099 +0000 UTC m=+4.231987836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:53.245836 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.245748    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 02:04:52 +0000 UTC" deadline="2028-01-12 18:15:50.086428923 +0000 UTC"
Jul 07 02:09:53.245836 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.245781    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13312h5m56.840651168s"
Jul 07 02:09:53.426995 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.426972    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:53.427127 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.427109    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:09:53.435798 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:53.435758    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde44aa3f_5598_4425_9d60_5b70370a9493.slice/crio-0f1763e0a0959e1a43aec830645daec09f04f70a219c678f427d24cd35a53d74 WatchSource:0}: Error finding container 0f1763e0a0959e1a43aec830645daec09f04f70a219c678f427d24cd35a53d74: Status 404 returned error can't find the container with id 0f1763e0a0959e1a43aec830645daec09f04f70a219c678f427d24cd35a53d74
Jul 07 02:09:53.437179 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:53.437149    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986445b3_5f4c_4975_9057_389f636b8f79.slice/crio-b5526165fe04ef7b849741df56c87032e367b46e17ad15139dd34cbdfe517ecf WatchSource:0}: Error finding container b5526165fe04ef7b849741df56c87032e367b46e17ad15139dd34cbdfe517ecf: Status 404 returned error can't find the container with id b5526165fe04ef7b849741df56c87032e367b46e17ad15139dd34cbdfe517ecf
Jul 07 02:09:53.439920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.439775    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9grv" event={"ID":"de44aa3f-5598-4425-9d60-5b70370a9493","Type":"ContainerStarted","Data":"0f1763e0a0959e1a43aec830645daec09f04f70a219c678f427d24cd35a53d74"}
Jul 07 02:09:53.440316 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:53.440288    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec11113_359b_4151_bd2b_cf5937c9879e.slice/crio-fc89d7fb9ef2a366b0fe00ab006c7d2ef0f579fabaa13220a768ec38a2c972cb WatchSource:0}: Error finding container fc89d7fb9ef2a366b0fe00ab006c7d2ef0f579fabaa13220a768ec38a2c972cb: Status 404 returned error can't find the container with id fc89d7fb9ef2a366b0fe00ab006c7d2ef0f579fabaa13220a768ec38a2c972cb
Jul 07 02:09:53.442937 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:53.442914    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8c127f_e39b_4e01_a740_8e5b4a5bf0c7.slice/crio-34dbe1d885357e3b56a246b6635316bb4352d48c152d47c72cadcf30d40f854f WatchSource:0}: Error finding container 34dbe1d885357e3b56a246b6635316bb4352d48c152d47c72cadcf30d40f854f: Status 404 returned error can't find the container with id 34dbe1d885357e3b56a246b6635316bb4352d48c152d47c72cadcf30d40f854f
Jul 07 02:09:53.444255 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:53.444236    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6144ddf_5525_45aa_afb7_ef69d27cab03.slice/crio-82377842dacf9c1bb6c110c8e9b986f1af53bd1d51b1805addb51e32b07eca14 WatchSource:0}: Error finding container 82377842dacf9c1bb6c110c8e9b986f1af53bd1d51b1805addb51e32b07eca14: Status 404 returned error can't find the container with id 82377842dacf9c1bb6c110c8e9b986f1af53bd1d51b1805addb51e32b07eca14
Jul 07 02:09:53.446278 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:09:53.445897    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd500a857_2643_4108_80af_8da289f0f3c7.slice/crio-2f736c569c9827705c842ea3a062b1ec09b7141b8ae3ca64cdb7c8f839f04360 WatchSource:0}: Error finding container 2f736c569c9827705c842ea3a062b1ec09b7141b8ae3ca64cdb7c8f839f04360: Status 404 returned error can't find the container with id 2f736c569c9827705c842ea3a062b1ec09b7141b8ae3ca64cdb7c8f839f04360
Jul 07 02:09:53.935263 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.934246    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:53.935263 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:53.934580    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:53.935263 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.934700    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:53.935263 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.934764    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:55.934744637 +0000 UTC m=+6.138165378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:53.935263 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.935142    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:53.935263 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:53.935188    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:55.935174545 +0000 UTC m=+6.138595280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:54.035378 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.035289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:54.035572 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:54.035455    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:09:54.035572 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:54.035476    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:09:54.035572 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:54.035488    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:54.035572 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:54.035559    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:56.035541474 +0000 UTC m=+6.238962212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:54.246764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.246651    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 02:04:52 +0000 UTC" deadline="2028-04-09 22:39:18.113780876 +0000 UTC"
Jul 07 02:09:54.246764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.246715    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15428h29m23.867069738s"
Jul 07 02:09:54.429952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.429917    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:54.430125 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:54.430056    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:09:54.430453 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.430431    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:54.430569 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:54.430549    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:09:54.474749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.474659    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"9021b5405cf9a0a1d182d23cf11338e2d49a059cc3dbf3e4baf0e274179bd1a7"}
Jul 07 02:09:54.484258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.483024    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j" event={"ID":"81581446-e6c0-4493-8636-61ae23ba83e2","Type":"ContainerStarted","Data":"1ddac39a5353d022d1755854f08aa18c6946bb631724686ceb2886dffcab08c6"}
Jul 07 02:09:54.488060 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.487876    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pd5z6" event={"ID":"986445b3-5f4c-4975-9057-389f636b8f79","Type":"ContainerStarted","Data":"b5526165fe04ef7b849741df56c87032e367b46e17ad15139dd34cbdfe517ecf"}
Jul 07 02:09:54.504111 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.504040    2568 generic.go:356] "Generic (PLEG): container finished" podID="fbaaf62e506e0ad292b5345120621ade" containerID="5ac2e707b75524f5e5df59f057970d68d34e26ec16264ec9f0e971c3b7bd2904" exitCode=0
Jul 07 02:09:54.504230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.504125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal" event={"ID":"fbaaf62e506e0ad292b5345120621ade","Type":"ContainerDied","Data":"5ac2e707b75524f5e5df59f057970d68d34e26ec16264ec9f0e971c3b7bd2904"}
Jul 07 02:09:54.512112 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.512081    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2wnp" event={"ID":"b6144ddf-5525-45aa-afb7-ef69d27cab03","Type":"ContainerStarted","Data":"82377842dacf9c1bb6c110c8e9b986f1af53bd1d51b1805addb51e32b07eca14"}
Jul 07 02:09:54.524397 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.524367    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mrpsn" event={"ID":"5eedf342-ae5c-45ce-bc64-474b56286922","Type":"ContainerStarted","Data":"f46ddc992d1a7a112a1583e07bb0f90bec65f8fe2878c8f3780cdbb8cf58a07b"}
Jul 07 02:09:54.533655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.533628    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r7s9h" event={"ID":"d500a857-2643-4108-80af-8da289f0f3c7","Type":"ContainerStarted","Data":"2f736c569c9827705c842ea3a062b1ec09b7141b8ae3ca64cdb7c8f839f04360"}
Jul 07 02:09:54.550407 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.550373    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerStarted","Data":"34dbe1d885357e3b56a246b6635316bb4352d48c152d47c72cadcf30d40f854f"}
Jul 07 02:09:54.567064 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.567025    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-57vj2" event={"ID":"bec11113-359b-4151-bd2b-cf5937c9879e","Type":"ContainerStarted","Data":"fc89d7fb9ef2a366b0fe00ab006c7d2ef0f579fabaa13220a768ec38a2c972cb"}
Jul 07 02:09:54.579121 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.579084    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal" event={"ID":"a5a6799705e4948b663271414d9e6b73","Type":"ContainerStarted","Data":"90a6a8c5b4252a87dacc9a7f3c7c5040ba0f94b3ba2dbfda50da6628a2aaafcf"}
Jul 07 02:09:54.594066 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:54.593999    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-152.ec2.internal" podStartSLOduration=3.593974788 podStartE2EDuration="3.593974788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:54.593965026 +0000 UTC m=+4.797385786" watchObservedRunningTime="2026-07-07 02:09:54.593974788 +0000 UTC m=+4.797395533"
Jul 07 02:09:55.427895 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:55.427856    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:55.428432 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:55.428009    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:09:55.593388 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:55.593335    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal" event={"ID":"fbaaf62e506e0ad292b5345120621ade","Type":"ContainerStarted","Data":"23bbdb978ae779c58a40269b8807ffb07046fb856f83905bb39f883c627232ac"}
Jul 07 02:09:55.608670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:55.608611    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-152.ec2.internal" podStartSLOduration=4.608588106 podStartE2EDuration="4.608588106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:55.608300721 +0000 UTC m=+5.811721466" watchObservedRunningTime="2026-07-07 02:09:55.608588106 +0000 UTC m=+5.812008851"
Jul 07 02:09:55.954080 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:55.954044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:55.954269 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:55.954108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:55.954332 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:55.954301    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:55.954382 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:55.954368    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:59.954346539 +0000 UTC m=+10.157767264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:55.954810 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:55.954790    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:55.954901 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:55.954843    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:59.954828611 +0000 UTC m=+10.158249346 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:56.054976 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:56.054933    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:56.055160 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:56.055072    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:09:56.055160 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:56.055086    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:09:56.055160 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:56.055095    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:56.055160 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:56.055148    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:00.055132136 +0000 UTC m=+10.258552865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:09:56.427796 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:56.427723    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:56.427956 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:56.427841    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:09:56.428292 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:56.428135    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:56.428292 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:56.428252    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:09:57.427751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:57.427714    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:57.427928 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:57.427854    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:09:58.428169 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:58.427601    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:58.428169 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:58.427621    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:09:58.428169 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:58.427726    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:09:58.428169 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:58.428122    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:09:59.427511 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:59.427462    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:59.427686 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:59.427643    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:09:59.985109 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:59.985051    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:09:59.985574 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:09:59.985153    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:09:59.985574 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:59.985277    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:59.985574 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:59.985314    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:09:59.985574 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:59.985351    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:07.985330314 +0000 UTC m=+18.188751040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:09:59.985574 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:09:59.985376    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:07.985359759 +0000 UTC m=+18.188780489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:10:00.086123 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:00.086055    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:00.086320 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:00.086277    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:10:00.086320 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:00.086297    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:10:00.086320 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:00.086309    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:10:00.086464 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:00.086366    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:08.086345165 +0000 UTC m=+18.289765907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:10:00.429055 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:00.428587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:00.429055 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:00.428692    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:00.429055 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:00.428853    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:00.429055 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:00.428939    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:01.427118 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:01.427089    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:01.427565 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:01.427231    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:02.428140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:02.428101    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:02.428140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:02.428126    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:02.428624 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:02.428242    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:02.428624 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:02.428325    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:03.427848 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:03.427812    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:03.428006 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:03.427975    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:04.427485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:04.427438    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:04.427943 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:04.427575    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:04.427943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:04.427639    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:04.427943 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:04.427768    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:05.427314 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:05.427281    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:05.427479 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:05.427392    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:06.427947 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:06.427909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:06.428400 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:06.428051    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:06.428400 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:06.428102    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:06.428400 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:06.428229    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:07.427440 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:07.427408    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:07.427667 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:07.427550    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:08.049958 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:08.049920    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:08.050403 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.050081    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:10:08.050403 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:08.050112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:08.050403 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.050175    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:24.050128543 +0000 UTC m=+34.253549264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:10:08.050403 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.050198    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:10:08.050403 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.050242    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:24.050230586 +0000 UTC m=+34.253651313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:10:08.151295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:08.151260    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:08.151524 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.151465    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:10:08.151524 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.151505    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:10:08.151524 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.151521    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:10:08.151741 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.151593    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:24.151569355 +0000 UTC m=+34.354990080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:10:08.427914 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:08.427830    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:08.428080 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.427969    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:08.428080 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:08.428036    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:08.428187 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:08.428143    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:09.427689 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:09.427650    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:09.428123 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:09.427782    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:10.428328 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:10.428277    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:10.428796 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:10.428399    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:10.428853 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:10.428804    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:10.428923 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:10.428905    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:11.427270 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:11.427237    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:11.427453 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:11.427384    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:12.427652 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.427326    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:12.428270 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.427394    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:12.428270 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:12.427729    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:12.428270 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:12.427832    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:12.631308 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.631282    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2wnp" event={"ID":"b6144ddf-5525-45aa-afb7-ef69d27cab03","Type":"ContainerStarted","Data":"3a7966628ef2f08f80e76edcb4457563c4e6203e464f236e3f5f7f8de75cf00b"}
Jul 07 02:10:12.632752 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.632720    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mrpsn" event={"ID":"5eedf342-ae5c-45ce-bc64-474b56286922","Type":"ContainerStarted","Data":"8a1d75ccc7641a1328e5e4053dd81f0a09a6fa4b0bbe44c8637c7b9adb0401c5"}
Jul 07 02:10:12.634201 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.634180    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r7s9h" event={"ID":"d500a857-2643-4108-80af-8da289f0f3c7","Type":"ContainerStarted","Data":"3ce3250dafb39795dbc45ef739afbfde942d3ff267f2e42b5049204624f80ff0"}
Jul 07 02:10:12.635771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.635747    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7" containerID="5dd57e8d41942667c3bad6f39ae930bb3970d043f66a6e5c58fb80504de90c67" exitCode=0
Jul 07 02:10:12.635860 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.635826    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerDied","Data":"5dd57e8d41942667c3bad6f39ae930bb3970d043f66a6e5c58fb80504de90c67"}
Jul 07 02:10:12.637411 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.637392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-57vj2" event={"ID":"bec11113-359b-4151-bd2b-cf5937c9879e","Type":"ContainerStarted","Data":"3bc92fee9062d1c5c513459cf6a6bad73b98943373de1fe3967cc054fec612df"}
Jul 07 02:10:12.640646 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.640624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"cd960d59ce2caebe7e2e68af6b80e1f14302c2fa0b4bddaba84239d31dee683e"}
Jul 07 02:10:12.640733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.640695    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"6970ef54d2ea70200afd42ba5d458f4db084df5fe9a2b3630fa70b43d8091a88"}
Jul 07 02:10:12.640779 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.640747    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"e7b35ba35886a2e290a10f57ab2745cd32eca2cec4601c4b96936d1dd14585ad"}
Jul 07 02:10:12.640811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.640795    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"ceaad32b94a7ca4ce4ffe0ccabeb13392669f706a343d2d61d7cbc65a5d11192"}
Jul 07 02:10:12.640846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.640837    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"28d060abcbf38d760d8c3572c835cc0e47403e0638541543d263aafef0efa3df"}
Jul 07 02:10:12.640876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.640867    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"23adba2b5f0154b46c9b76d38b940aa96928c23678243a06191998f91dbd4634"}
Jul 07 02:10:12.642431 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.642413    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j" event={"ID":"81581446-e6c0-4493-8636-61ae23ba83e2","Type":"ContainerStarted","Data":"ae4880a8deb659f35919ecd1c71d58a804b43e08be51a2855eed3c2d33ca8c10"}
Jul 07 02:10:12.643718 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.643699    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pd5z6" event={"ID":"986445b3-5f4c-4975-9057-389f636b8f79","Type":"ContainerStarted","Data":"0a48ca96503eba00c8b79f6ea25c63ee26f238ffb5f1ad9fa9715b9192fc18ba"}
Jul 07 02:10:12.647183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.647139    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l2wnp" podStartSLOduration=4.190553466 podStartE2EDuration="22.647123646s" totalImagesPullingTime="18.45657018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:12.647027623 +0000 UTC m=+22.850448367" watchObservedRunningTime="2026-07-07 02:10:12.647123646 +0000 UTC m=+22.850544391"
Jul 07 02:10:12.665345 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.665275    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pd5z6" podStartSLOduration=4.19782455 podStartE2EDuration="22.665262947s" totalImagesPullingTime="18.467438397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:12.6647516 +0000 UTC m=+22.868172343" watchObservedRunningTime="2026-07-07 02:10:12.665262947 +0000 UTC m=+22.868683698"
Jul 07 02:10:12.687177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.687126    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mrpsn" podStartSLOduration=4.181496143 podStartE2EDuration="22.687109467s" totalImagesPullingTime="18.505613324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:12.68667209 +0000 UTC m=+22.890092835" watchObservedRunningTime="2026-07-07 02:10:12.687109467 +0000 UTC m=+22.890530210"
Jul 07 02:10:12.707176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.707115    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r7s9h" podStartSLOduration=4.251777865 podStartE2EDuration="22.707098582s" totalImagesPullingTime="18.455320717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:12.70661622 +0000 UTC m=+22.910036966" watchObservedRunningTime="2026-07-07 02:10:12.707098582 +0000 UTC m=+22.910519327"
Jul 07 02:10:12.722412 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.722366    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-57vj2" podStartSLOduration=8.631240928 podStartE2EDuration="22.722351129s" totalImagesPullingTime="14.091110201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:12.721701152 +0000 UTC m=+22.925121898" watchObservedRunningTime="2026-07-07 02:10:12.722351129 +0000 UTC m=+22.925771894"
Jul 07 02:10:12.834188 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.834148    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:12.890920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:12.890866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:13.427648 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.427622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:13.427790 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:13.427757    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:13.432620 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.432595    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 02:10:13.648558 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.648464    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j" event={"ID":"81581446-e6c0-4493-8636-61ae23ba83e2","Type":"ContainerStarted","Data":"d5e318c1c8ac23a2a14fb254399af70231cd2cadf56454d02d58d271feaf6004"}
Jul 07 02:10:13.650031 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.649951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9grv" event={"ID":"de44aa3f-5598-4425-9d60-5b70370a9493","Type":"ContainerStarted","Data":"d753c30f1d6a7b3b76ca1e624d4ba71bd8e03e097b2b7f5eb3345950419ba8d4"}
Jul 07 02:10:13.664617 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.664567    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z9grv" podStartSLOduration=5.199404308 podStartE2EDuration="23.664553499s" totalImagesPullingTime="18.465149191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:13.664288997 +0000 UTC m=+23.867709743" watchObservedRunningTime="2026-07-07 02:10:13.664553499 +0000 UTC m=+23.867974243"
Jul 07 02:10:13.783628 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.783583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:13.837320 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:13.837286    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:14.322958 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:14.322844    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T02:10:13.432616166Z","UUID":"228a11bd-9361-4fec-9f21-376c533d5089","Handler":null,"Name":"","Endpoint":""}
Jul 07 02:10:14.324598 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:14.324576    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 02:10:14.324735 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:14.324610    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 02:10:14.426987 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:14.426950    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:14.427237 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:14.426964    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:14.427237 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:14.427090    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:14.427237 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:14.427172    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:15.427251 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:15.427008    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:15.427693 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:15.427369    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:15.559763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:15.559731    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:15.622312 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:15.622274    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:15.657092 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:15.657059    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"c43f98d43d2f11ea2f373a30f2f91b1d5aed03288e33a7bccefa820314477d39"}
Jul 07 02:10:15.658881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:15.658854    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j" event={"ID":"81581446-e6c0-4493-8636-61ae23ba83e2","Type":"ContainerStarted","Data":"ef6f01f6dc1c569bccdfc103af40abbd613e7d9baa3d42d9d27be7af81d7234f"}
Jul 07 02:10:15.677931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:15.677849    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wq6j" podStartSLOduration=4.797991891 podStartE2EDuration="25.677831733s" totalImagesPullingTime="20.879839842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:15.67737381 +0000 UTC m=+25.880794577" watchObservedRunningTime="2026-07-07 02:10:15.677831733 +0000 UTC m=+25.881252497"
Jul 07 02:10:16.427809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:16.427776    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:16.427809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:16.427806    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:16.428374 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:16.427907    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:16.428374 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:16.428034    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:17.035166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.035116    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:10:17.035807 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.035778    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:10:17.427063 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.427023    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:17.427193 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:17.427131    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:17.666545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.666317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerStarted","Data":"7532dd15dd93601e17c3c188df138933d43754d176cd3b6362b4bdac4c6f9226"}
Jul 07 02:10:17.670155 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.670121    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" event={"ID":"ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e","Type":"ContainerStarted","Data":"29f4e11161c6795d9534a412171ca0731435a5e97ceaa2ee3f1b95cc6b41ffe1"}
Jul 07 02:10:17.670469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.670403    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:10:17.670469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.670441    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:10:17.670634 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.670514    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:10:17.670634 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.670561    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:10:17.670970 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.670938    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-57vj2"
Jul 07 02:10:17.685482 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.685462    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:10:17.685646 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.685635    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:10:17.715195 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.715078    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55" podStartSLOduration=9.187219488 podStartE2EDuration="27.715058278s" totalImagesPullingTime="18.52783879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:17.712530598 +0000 UTC m=+27.915951342" watchObservedRunningTime="2026-07-07 02:10:17.715058278 +0000 UTC m=+27.918479026"
Jul 07 02:10:17.774118 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:17.774088    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:18.427943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:18.427906    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:18.428151 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:18.428010    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:18.428151 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:18.428071    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:18.428259 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:18.428172    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:18.674272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:18.674230    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7" containerID="7532dd15dd93601e17c3c188df138933d43754d176cd3b6362b4bdac4c6f9226" exitCode=0
Jul 07 02:10:18.674728 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:18.674292    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerDied","Data":"7532dd15dd93601e17c3c188df138933d43754d176cd3b6362b4bdac4c6f9226"}
Jul 07 02:10:18.783262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:18.783171    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:19.070077 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.070001    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:19.143309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.143277    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:19.308023 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.307990    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q9bb5"]
Jul 07 02:10:19.308165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.308095    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:19.308218 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:19.308194    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:19.310612 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.310592    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-slv96"]
Jul 07 02:10:19.310718 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.310663    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:19.310762 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:19.310740    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:19.311437 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.311400    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mzcc5"]
Jul 07 02:10:19.311564 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:19.311508    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:19.311630 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:19.311584    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:20.428681 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:20.428479    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:20.429148 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:20.428775    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:20.680476 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:20.680394    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7" containerID="35af1319e01079628ecd73a7dd089942bdbc49bdb88a711c568c994f0c21a846" exitCode=0
Jul 07 02:10:20.680476 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:20.680449    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerDied","Data":"35af1319e01079628ecd73a7dd089942bdbc49bdb88a711c568c994f0c21a846"}
Jul 07 02:10:21.427743 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:21.427665    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:21.427916 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:21.427671    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:21.427916 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:21.427799    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:21.427916 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:21.427864    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:22.427098 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:22.427050    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:22.427738 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:22.427175    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:22.686975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:22.686893    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7" containerID="6c4ace393f1d32f211fd1609c86f3855fd55db8127c92052bb0ea7185b82c6fc" exitCode=0
Jul 07 02:10:22.686975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:22.686952    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerDied","Data":"6c4ace393f1d32f211fd1609c86f3855fd55db8127c92052bb0ea7185b82c6fc"}
Jul 07 02:10:23.221783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:23.221746    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:23.324057 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:23.323997    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:23.427670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:23.427583    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:23.428019 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:23.427586    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:23.428019 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:23.427701    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q9bb5" podUID="a4358822-184a-4fa7-8d72-27b1af7c6d75"
Jul 07 02:10:23.428019 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:23.427768    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slv96" podUID="f321a9f9-1646-4965-99ab-2805638e6c10"
Jul 07 02:10:24.064489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.064456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:24.064696 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.064667    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:10:24.064748 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.064695    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:24.064748 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.064728    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret podName:a4358822-184a-4fa7-8d72-27b1af7c6d75 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:56.064711684 +0000 UTC m=+66.268132407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret") pod "global-pull-secret-syncer-q9bb5" (UID: "a4358822-184a-4fa7-8d72-27b1af7c6d75") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:10:24.064845 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.064791    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:10:24.064845 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.064839    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:56.064827138 +0000 UTC m=+66.268247865 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:10:24.165599 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.165558    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:24.165783 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.165750    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:10:24.165783 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.165774    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:10:24.165890 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.165787    2568 projected.go:196] Error preparing data for projected volume kube-api-access-nzddh for pod openshift-network-diagnostics/network-check-target-slv96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:10:24.165890 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.165853    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh podName:f321a9f9-1646-4965-99ab-2805638e6c10 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:56.165833378 +0000 UTC m=+66.369254121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nzddh" (UniqueName: "kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh") pod "network-check-target-slv96" (UID: "f321a9f9-1646-4965-99ab-2805638e6c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:10:24.427851 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.427826    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:24.428292 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:24.427942    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:10:24.911855 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.911826    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-152.ec2.internal" event="NodeReady"
Jul 07 02:10:24.912049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.912009    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 02:10:24.956125 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.956093    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9zcvr"]
Jul 07 02:10:24.958381 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.958352    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xt4rs"]
Jul 07 02:10:24.958537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.958451    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:24.960506 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.960467    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:24.961295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.961272    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 02:10:24.961401 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.961390    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tv6md\""
Jul 07 02:10:24.961721 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.961705    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 02:10:24.963106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.963067    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gjz6q\""
Jul 07 02:10:24.963106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.963100    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 02:10:24.963264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.963100    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 02:10:24.963264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.963074    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 02:10:24.972698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.970305    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9zcvr"]
Jul 07 02:10:24.974701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:24.974645    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xt4rs"]
Jul 07 02:10:25.070966 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.070900    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:25.073840 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.073810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.073948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.073847    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a387479e-f772-49bc-b157-c146c9358d3e-cert\") pod \"ingress-canary-xt4rs\" (UID: \"a387479e-f772-49bc-b157-c146c9358d3e\") " pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.073948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.073868    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcsmj\" (UniqueName: \"kubernetes.io/projected/a387479e-f772-49bc-b157-c146c9358d3e-kube-api-access-dcsmj\") pod \"ingress-canary-xt4rs\" (UID: \"a387479e-f772-49bc-b157-c146c9358d3e\") " pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.074030 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.073993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5a8d96c-2c84-479e-a76c-de376aca0782-config-volume\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.074111 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.074095    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cgwz\" (UniqueName: \"kubernetes.io/projected/a5a8d96c-2c84-479e-a76c-de376aca0782-kube-api-access-2cgwz\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.074168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.074141    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a8d96c-2c84-479e-a76c-de376aca0782-tmp-dir\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.175467 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175368    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a8d96c-2c84-479e-a76c-de376aca0782-tmp-dir\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.175467 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175429    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.175467 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a387479e-f772-49bc-b157-c146c9358d3e-cert\") pod \"ingress-canary-xt4rs\" (UID: \"a387479e-f772-49bc-b157-c146c9358d3e\") " pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.175763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175481    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcsmj\" (UniqueName: \"kubernetes.io/projected/a387479e-f772-49bc-b157-c146c9358d3e-kube-api-access-dcsmj\") pod \"ingress-canary-xt4rs\" (UID: \"a387479e-f772-49bc-b157-c146c9358d3e\") " pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.175763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5a8d96c-2c84-479e-a76c-de376aca0782-config-volume\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.175763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cgwz\" (UniqueName: \"kubernetes.io/projected/a5a8d96c-2c84-479e-a76c-de376aca0782-kube-api-access-2cgwz\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.175763 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:25.175613    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:25.175763 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:25.175688    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:25.675665781 +0000 UTC m=+35.879086503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:10:25.175763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.175746    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a8d96c-2c84-479e-a76c-de376aca0782-tmp-dir\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.176191 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.176173    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5a8d96c-2c84-479e-a76c-de376aca0782-config-volume\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.179645 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.179627    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a387479e-f772-49bc-b157-c146c9358d3e-cert\") pod \"ingress-canary-xt4rs\" (UID: \"a387479e-f772-49bc-b157-c146c9358d3e\") " pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.185214 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.185191    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cgwz\" (UniqueName: \"kubernetes.io/projected/a5a8d96c-2c84-479e-a76c-de376aca0782-kube-api-access-2cgwz\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.185330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.185247    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcsmj\" (UniqueName: \"kubernetes.io/projected/a387479e-f772-49bc-b157-c146c9358d3e-kube-api-access-dcsmj\") pod \"ingress-canary-xt4rs\" (UID: \"a387479e-f772-49bc-b157-c146c9358d3e\") " pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.283670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.283634    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xt4rs"
Jul 07 02:10:25.427271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.427028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:25.427271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.427028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:25.430167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.430141    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g22vd\""
Jul 07 02:10:25.430167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.430158    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 02:10:25.430552 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.430198    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 02:10:25.430552 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.430158    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 02:10:25.432149 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.432130    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xt4rs"]
Jul 07 02:10:25.435870 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:10:25.435848    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda387479e_f772_49bc_b157_c146c9358d3e.slice/crio-bc73307db7815596095917814bda970776d108ab4384a8639ddf3d508342fb60 WatchSource:0}: Error finding container bc73307db7815596095917814bda970776d108ab4384a8639ddf3d508342fb60: Status 404 returned error can't find the container with id bc73307db7815596095917814bda970776d108ab4384a8639ddf3d508342fb60
Jul 07 02:10:25.680399 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.680320    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:25.680566 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:25.680466    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:25.680566 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:25.680543    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:26.680526212 +0000 UTC m=+36.883946946 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:10:25.694198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:25.694168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xt4rs" event={"ID":"a387479e-f772-49bc-b157-c146c9358d3e","Type":"ContainerStarted","Data":"bc73307db7815596095917814bda970776d108ab4384a8639ddf3d508342fb60"}
Jul 07 02:10:26.427469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:26.427430    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:26.431808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:26.431772    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 02:10:26.432270 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:26.431837    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q64zw\""
Jul 07 02:10:26.690025 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:26.689941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:26.690195 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:26.690115    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:26.690195 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:26.690187    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:28.690165134 +0000 UTC m=+38.893585857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:10:28.705798 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:28.705766    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:28.706166 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:28.705906    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:28.706166 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:28.705958    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:32.705942748 +0000 UTC m=+42.909363474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:10:29.059943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:29.059698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:29.130056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:29.130023    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:29.706797 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:29.706761    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7" containerID="44c214f3216e72cc3148930345622a8eb5f3e06beebfd816c580a447822ef9f9" exitCode=0
Jul 07 02:10:29.707184 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:29.706842    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerDied","Data":"44c214f3216e72cc3148930345622a8eb5f3e06beebfd816c580a447822ef9f9"}
Jul 07 02:10:29.708223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:29.708201    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xt4rs" event={"ID":"a387479e-f772-49bc-b157-c146c9358d3e","Type":"ContainerStarted","Data":"4e891240049ca0bdb355b64654b278ab2056c57f1382e7734b66a3516f034ac5"}
Jul 07 02:10:29.747587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:29.747542    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xt4rs" podStartSLOduration=2.372200057 podStartE2EDuration="5.747525426s" totalImagesPullingTime="3.375325369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:29.747239487 +0000 UTC m=+39.950660233" watchObservedRunningTime="2026-07-07 02:10:29.747525426 +0000 UTC m=+39.950946170"
Jul 07 02:10:30.713139 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:30.713106    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7" containerID="370b72a104c660e22cdef18cddcb155e8cba9d64ae1346cebc61c43c56a26dca" exitCode=0
Jul 07 02:10:30.713534 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:30.713199    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerDied","Data":"370b72a104c660e22cdef18cddcb155e8cba9d64ae1346cebc61c43c56a26dca"}
Jul 07 02:10:31.527604 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:31.527573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:31.628165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:31.628121    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:31.718456 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:31.718417    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" event={"ID":"ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7","Type":"ContainerStarted","Data":"1b16c89bed2dffe28e95a19ed05f1e22615b001ec68703b36395a5bd7b66eb54"}
Jul 07 02:10:31.741832 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:31.741787    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xrrzv" podStartSLOduration=10.102744482 podStartE2EDuration="41.741766128s" totalImagesPullingTime="31.639021646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:31.739961192 +0000 UTC m=+41.943381937" watchObservedRunningTime="2026-07-07 02:10:31.741766128 +0000 UTC m=+41.945186872"
Jul 07 02:10:31.860722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:31.860619    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:31.962879 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:31.962835    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:32.062018 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:32.061983    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:32.112825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:32.112731    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:32.734699 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:32.734660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:32.735089 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:32.734771    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:32.735089 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:32.734830    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:40.734811751 +0000 UTC m=+50.938232494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:10:40.795540 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:40.795483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:40.795999 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:40.795635    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:40.795999 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:40.795709    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:56.795692154 +0000 UTC m=+66.999112877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:10:44.207390 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:44.207351    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:44.271782 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:44.271750    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:49.687186 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:49.687157    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9t55"
Jul 07 02:10:49.801906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:49.801877    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:49.861115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:49.861088    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:10:56.107731 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.107677    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:56.108304 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.107766    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:10:56.110471 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.110450    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 02:10:56.110584 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.110489    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 02:10:56.118623 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:56.118600    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 02:10:56.118702 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:56.118664    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:12:00.11864373 +0000 UTC m=+130.322064453 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : secret "metrics-daemon-secret" not found
Jul 07 02:10:56.122082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.122065    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4358822-184a-4fa7-8d72-27b1af7c6d75-original-pull-secret\") pod \"global-pull-secret-syncer-q9bb5\" (UID: \"a4358822-184a-4fa7-8d72-27b1af7c6d75\") " pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:56.209049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.209000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:56.211676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.211658    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 02:10:56.221939 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.221926    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 02:10:56.232127 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.232108    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzddh\" (UniqueName: \"kubernetes.io/projected/f321a9f9-1646-4965-99ab-2805638e6c10-kube-api-access-nzddh\") pod \"network-check-target-slv96\" (UID: \"f321a9f9-1646-4965-99ab-2805638e6c10\") " pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:56.342224 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.342197    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g22vd\""
Jul 07 02:10:56.343256 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.343234    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q9bb5"
Jul 07 02:10:56.350223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.350204    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:10:56.474001 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.473980    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q9bb5"]
Jul 07 02:10:56.476551 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:10:56.476525    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4358822_184a_4fa7_8d72_27b1af7c6d75.slice/crio-465b5118641919cd5c5b568a367cc19ab1deec461097c7165339b9b50bedd7ff WatchSource:0}: Error finding container 465b5118641919cd5c5b568a367cc19ab1deec461097c7165339b9b50bedd7ff: Status 404 returned error can't find the container with id 465b5118641919cd5c5b568a367cc19ab1deec461097c7165339b9b50bedd7ff
Jul 07 02:10:56.489604 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.489577    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-slv96"]
Jul 07 02:10:56.492923 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:10:56.492900    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf321a9f9_1646_4965_99ab_2805638e6c10.slice/crio-c5411546960d2826d3934464f5cdc66bf923aba1db5f82a58f6a29efb01939f0 WatchSource:0}: Error finding container c5411546960d2826d3934464f5cdc66bf923aba1db5f82a58f6a29efb01939f0: Status 404 returned error can't find the container with id c5411546960d2826d3934464f5cdc66bf923aba1db5f82a58f6a29efb01939f0
Jul 07 02:10:56.775808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.775763    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-slv96" event={"ID":"f321a9f9-1646-4965-99ab-2805638e6c10","Type":"ContainerStarted","Data":"c5411546960d2826d3934464f5cdc66bf923aba1db5f82a58f6a29efb01939f0"}
Jul 07 02:10:56.776676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.776655    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q9bb5" event={"ID":"a4358822-184a-4fa7-8d72-27b1af7c6d75","Type":"ContainerStarted","Data":"465b5118641919cd5c5b568a367cc19ab1deec461097c7165339b9b50bedd7ff"}
Jul 07 02:10:56.813280 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:10:56.813248    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:10:56.813429 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:56.813409    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:10:56.813488 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:10:56.813478    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:11:28.813461469 +0000 UTC m=+99.016882191 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:11:01.792684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:01.792643    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q9bb5" event={"ID":"a4358822-184a-4fa7-8d72-27b1af7c6d75","Type":"ContainerStarted","Data":"b3a0205b8f9cc792187bc50d79d04c8cd8e199cfb52d829c3360e133a49ebbb8"}
Jul 07 02:11:01.794081 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:01.794057    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-slv96" event={"ID":"f321a9f9-1646-4965-99ab-2805638e6c10","Type":"ContainerStarted","Data":"17c193ba4f4e07b4e659df583c4b98911fe80ea151fe7cde87e9b32ab1398673"}
Jul 07 02:11:01.794192 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:01.794180    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:11:01.808261 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:01.808214    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q9bb5" podStartSLOduration=67.126467266 podStartE2EDuration="1m11.808200879s" totalImagesPullingTime="4.681733613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:11:01.807174154 +0000 UTC m=+72.010594898" watchObservedRunningTime="2026-07-07 02:11:01.808200879 +0000 UTC m=+72.011621601"
Jul 07 02:11:01.821269 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:01.821227    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-slv96" podStartSLOduration=67.159126415 podStartE2EDuration="1m11.821213857s" totalImagesPullingTime="4.662087442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:11:01.820523306 +0000 UTC m=+72.023944044" watchObservedRunningTime="2026-07-07 02:11:01.821213857 +0000 UTC m=+72.024634578"
Jul 07 02:11:28.831617 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:28.831581    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:11:28.832098 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:11:28.831701    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:11:28.832098 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:11:28.831760    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:12:32.831747839 +0000 UTC m=+163.035168562 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:11:32.798669 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:11:32.798637    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-slv96"
Jul 07 02:12:00.139620 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:12:00.139567    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:12:00.140115 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:12:00.139717    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 02:12:00.140115 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:12:00.139795    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:02.139778301 +0000 UTC m=+252.343199024 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : secret "metrics-daemon-secret" not found
Jul 07 02:12:27.977992 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:12:27.977947    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9zcvr" podUID="a5a8d96c-2c84-479e-a76c-de376aca0782"
Jul 07 02:12:27.994813 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:12:27.994787    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:12:29.439064 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:12:29.439017    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:12:32.866325 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:12:32.866291    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:12:32.866751 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:12:32.866470    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:12:32.866751 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:12:32.866586    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls podName:a5a8d96c-2c84-479e-a76c-de376aca0782 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:34.866561681 +0000 UTC m=+285.069982426 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls") pod "dns-default-9zcvr" (UID: "a5a8d96c-2c84-479e-a76c-de376aca0782") : secret "dns-default-metrics-tls" not found
Jul 07 02:12:43.426944 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:12:43.426867    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:13:23.989170 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:23.989138    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:13:33.792427 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:33.792389    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:13:57.931485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.931453    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-sdhnx"]
Jul 07 02:13:57.933150 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.933134    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:57.934730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.934702    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7prkb"]
Jul 07 02:13:57.935719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.935693    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 02:13:57.935936 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.935913    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:13:57.936036 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.935969    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 02:13:57.936222 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.936207    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:57.936777 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.936761    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 02:13:57.936893 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.936805    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5tm4g\""
Jul 07 02:13:57.939263 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.939243    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 02:13:57.939363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.939275    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 02:13:57.939363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.939328    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 02:13:57.939637 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.939618    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 02:13:57.939790 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.939770    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pwclk\""
Jul 07 02:13:57.944074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.944056    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 02:13:57.945805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.945788    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 02:13:57.965592 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.965565    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7prkb"]
Jul 07 02:13:57.969246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:57.969220    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-sdhnx"]
Jul 07 02:13:58.025896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.025858    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"]
Jul 07 02:13:58.027676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.027654    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"
Jul 07 02:13:58.029813 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.029786    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"]
Jul 07 02:13:58.030089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.030073    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-chbgc\""
Jul 07 02:13:58.031266 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.031247    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"]
Jul 07 02:13:58.031410 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.031391    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.032989 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.032971    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75fbfc4f44-6r7cr"]
Jul 07 02:13:58.033100 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.033083    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.033816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.033796    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 02:13:58.033816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.033810    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 02:13:58.034011 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.033833    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wv5t5\""
Jul 07 02:13:58.034153 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.034137    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 02:13:58.034259 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.034245    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:13:58.034960 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.034944    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.035684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.035666    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 02:13:58.035877 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.035863    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 02:13:58.035945 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.035866    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:13:58.036002 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.035958    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lqkc7\""
Jul 07 02:13:58.037334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.037318    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 02:13:58.037406 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.037333    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 02:13:58.037692 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.037677    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 02:13:58.037974 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.037959    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2bvdv\""
Jul 07 02:13:58.042535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.042516    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"]
Jul 07 02:13:58.043613 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45lq7\" (UniqueName: \"kubernetes.io/projected/45f4fec7-722a-4b66-98c1-185c65ad2816-kube-api-access-45lq7\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.043702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45f4fec7-722a-4b66-98c1-185c65ad2816-serving-cert\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.043702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043656    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 02:13:58.043702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043681    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45f4fec7-722a-4b66-98c1-185c65ad2816-config\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.043818 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043719    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-tmp\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.043818 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043743    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvtv7\" (UniqueName: \"kubernetes.io/projected/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-kube-api-access-zvtv7\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.043818 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043800    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-snapshots\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.043931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-serving-cert\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.043931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043879    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f4fec7-722a-4b66-98c1-185c65ad2816-trusted-ca\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.044003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.044003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.043964    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.046181 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.046162    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"]
Jul 07 02:13:58.049946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.049914    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75fbfc4f44-6r7cr"]
Jul 07 02:13:58.061180 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.061162    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"]
Jul 07 02:13:58.144916 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.144879    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ff63f40-bce3-427c-990a-9545b6cc9d5b-config\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.144916 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.144911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.144929    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-installation-pull-secrets\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.144948    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb7gx\" (UniqueName: \"kubernetes.io/projected/44a6ee7e-b1aa-4763-b5da-3e50450efa44-kube-api-access-vb7gx\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.144969    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-certificates\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45lq7\" (UniqueName: \"kubernetes.io/projected/45f4fec7-722a-4b66-98c1-185c65ad2816-kube-api-access-45lq7\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45f4fec7-722a-4b66-98c1-185c65ad2816-config\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145093    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-image-registry-private-configuration\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145114    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdpcn\" (UniqueName: \"kubernetes.io/projected/3ff63f40-bce3-427c-990a-9545b6cc9d5b-kube-api-access-jdpcn\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145141    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-tmp\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvtv7\" (UniqueName: \"kubernetes.io/projected/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-kube-api-access-zvtv7\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145174    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfsgh\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-kube-api-access-hfsgh\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145192    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f4fec7-722a-4b66-98c1-185c65ad2816-trusted-ca\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145225    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ff63f40-bce3-427c-990a-9545b6cc9d5b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.145424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145370    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145473    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-bound-sa-token\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145536    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45f4fec7-722a-4b66-98c1-185c65ad2816-serving-cert\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145560    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d76295-5133-4a09-a23b-18ec6b3ed6fb-ca-trust-extracted\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145583    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsz64\" (UniqueName: \"kubernetes.io/projected/09b78ba0-b49e-4412-b3e8-98d48570bb0a-kube-api-access-tsz64\") pod \"network-check-source-989f8b8dc-zr2kl\" (UID: \"09b78ba0-b49e-4412-b3e8-98d48570bb0a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-trusted-ca\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145650    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-snapshots\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.145825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145675    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-serving-cert\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.146157 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145881    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-tmp\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.146157 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.145967    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45f4fec7-722a-4b66-98c1-185c65ad2816-config\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.146157 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.146015    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.146157 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.146110    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f4fec7-722a-4b66-98c1-185c65ad2816-trusted-ca\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.146294 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.146207    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.146441 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.146425    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-snapshots\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.147915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.147898    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-serving-cert\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.147999 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.147964    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45f4fec7-722a-4b66-98c1-185c65ad2816-serving-cert\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.154149 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.154119    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvtv7\" (UniqueName: \"kubernetes.io/projected/a77aff70-34c1-4ab4-a4be-901ecebc8a7c-kube-api-access-zvtv7\") pod \"insights-operator-5c847dcfc4-7prkb\" (UID: \"a77aff70-34c1-4ab4-a4be-901ecebc8a7c\") " pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.154149 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.154136    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45lq7\" (UniqueName: \"kubernetes.io/projected/45f4fec7-722a-4b66-98c1-185c65ad2816-kube-api-access-45lq7\") pod \"console-operator-68b757fb67-sdhnx\" (UID: \"45f4fec7-722a-4b66-98c1-185c65ad2816\") " pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.206424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.206354    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:13:58.244256 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.244223    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:13:58.246483 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246446    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdpcn\" (UniqueName: \"kubernetes.io/projected/3ff63f40-bce3-427c-990a-9545b6cc9d5b-kube-api-access-jdpcn\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.246638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfsgh\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-kube-api-access-hfsgh\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.246638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.246638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246561    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ff63f40-bce3-427c-990a-9545b6cc9d5b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.246638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246585    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-bound-sa-token\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.246638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246604    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d76295-5133-4a09-a23b-18ec6b3ed6fb-ca-trust-extracted\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.246638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246628    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsz64\" (UniqueName: \"kubernetes.io/projected/09b78ba0-b49e-4412-b3e8-98d48570bb0a-kube-api-access-tsz64\") pod \"network-check-source-989f8b8dc-zr2kl\" (UID: \"09b78ba0-b49e-4412-b3e8-98d48570bb0a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246665    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-trusted-ca\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.246680    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ff63f40-bce3-427c-990a-9545b6cc9d5b-config\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.246740    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls podName:44a6ee7e-b1aa-4763-b5da-3e50450efa44 nodeName:}" failed. No retries permitted until 2026-07-07 02:13:58.746721942 +0000 UTC m=+248.950142664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6b7pj" (UID: "44a6ee7e-b1aa-4763-b5da-3e50450efa44") : secret "samples-operator-tls" not found
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-installation-pull-secrets\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246839    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb7gx\" (UniqueName: \"kubernetes.io/projected/44a6ee7e-b1aa-4763-b5da-3e50450efa44-kube-api-access-vb7gx\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246869    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-certificates\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.246906    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-image-registry-private-configuration\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.247161    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d76295-5133-4a09-a23b-18ec6b3ed6fb-ca-trust-extracted\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.247387    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ff63f40-bce3-427c-990a-9545b6cc9d5b-config\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.247885 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.247726    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:13:58.247885 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.247740    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75fbfc4f44-6r7cr: secret "image-registry-tls" not found
Jul 07 02:13:58.247885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.247737    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-trusted-ca\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.247885 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.247782    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls podName:58d76295-5133-4a09-a23b-18ec6b3ed6fb nodeName:}" failed. No retries permitted until 2026-07-07 02:13:58.747766033 +0000 UTC m=+248.951186755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls") pod "image-registry-75fbfc4f44-6r7cr" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb") : secret "image-registry-tls" not found
Jul 07 02:13:58.248144 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.247918    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-certificates\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.248925 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.248906    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb"
Jul 07 02:13:58.249421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.249402    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ff63f40-bce3-427c-990a-9545b6cc9d5b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.249531 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.249516    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-image-registry-private-configuration\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.249588 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.249566    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-installation-pull-secrets\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.256698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.256517    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdpcn\" (UniqueName: \"kubernetes.io/projected/3ff63f40-bce3-427c-990a-9545b6cc9d5b-kube-api-access-jdpcn\") pod \"service-ca-operator-5d46d9ffd4-4tnxr\" (UID: \"3ff63f40-bce3-427c-990a-9545b6cc9d5b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.256698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.256561    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb7gx\" (UniqueName: \"kubernetes.io/projected/44a6ee7e-b1aa-4763-b5da-3e50450efa44-kube-api-access-vb7gx\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.257133 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.257060    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-bound-sa-token\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.258053 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.258012    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfsgh\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-kube-api-access-hfsgh\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.258137 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.258063    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsz64\" (UniqueName: \"kubernetes.io/projected/09b78ba0-b49e-4412-b3e8-98d48570bb0a-kube-api-access-tsz64\") pod \"network-check-source-989f8b8dc-zr2kl\" (UID: \"09b78ba0-b49e-4412-b3e8-98d48570bb0a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"
Jul 07 02:13:58.308869 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.308833    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:13:58.337811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.337782    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"
Jul 07 02:13:58.344693 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.344669    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"
Jul 07 02:13:58.375442 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.375410    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-sdhnx"]
Jul 07 02:13:58.381632 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:13:58.381592    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f4fec7_722a_4b66_98c1_185c65ad2816.slice/crio-58e1e4105265e4f4a205f37a38973933239bfb84f3317b53e6bf98f76d199642 WatchSource:0}: Error finding container 58e1e4105265e4f4a205f37a38973933239bfb84f3317b53e6bf98f76d199642: Status 404 returned error can't find the container with id 58e1e4105265e4f4a205f37a38973933239bfb84f3317b53e6bf98f76d199642
Jul 07 02:13:58.396947 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.396874    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7prkb"]
Jul 07 02:13:58.401237 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:13:58.401054    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77aff70_34c1_4ab4_a4be_901ecebc8a7c.slice/crio-6ddd3d72837cdfc272e46719186ce0014233b08727b9bc4107d3f2126315186a WatchSource:0}: Error finding container 6ddd3d72837cdfc272e46719186ce0014233b08727b9bc4107d3f2126315186a: Status 404 returned error can't find the container with id 6ddd3d72837cdfc272e46719186ce0014233b08727b9bc4107d3f2126315186a
Jul 07 02:13:58.480331 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.480259    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl"]
Jul 07 02:13:58.483045 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:13:58.483017    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b78ba0_b49e_4412_b3e8_98d48570bb0a.slice/crio-fde49b522dc99eaa55a47359a9434bb6b1e0bd259d860354ed502e36ec40312d WatchSource:0}: Error finding container fde49b522dc99eaa55a47359a9434bb6b1e0bd259d860354ed502e36ec40312d: Status 404 returned error can't find the container with id fde49b522dc99eaa55a47359a9434bb6b1e0bd259d860354ed502e36ec40312d
Jul 07 02:13:58.492143 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.492119    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr"]
Jul 07 02:13:58.494801 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:13:58.494776    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff63f40_bce3_427c_990a_9545b6cc9d5b.slice/crio-bef0f3cdbb7be13a317a8bf7e21cf726a724347e27eefc1e02f46beb6f919cf5 WatchSource:0}: Error finding container bef0f3cdbb7be13a317a8bf7e21cf726a724347e27eefc1e02f46beb6f919cf5: Status 404 returned error can't find the container with id bef0f3cdbb7be13a317a8bf7e21cf726a724347e27eefc1e02f46beb6f919cf5
Jul 07 02:13:58.750312 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.750276    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:58.750517 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:58.750342    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:58.750517 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.750430    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 02:13:58.750517 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.750512    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls podName:44a6ee7e-b1aa-4763-b5da-3e50450efa44 nodeName:}" failed. No retries permitted until 2026-07-07 02:13:59.750476009 +0000 UTC m=+249.953896731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6b7pj" (UID: "44a6ee7e-b1aa-4763-b5da-3e50450efa44") : secret "samples-operator-tls" not found
Jul 07 02:13:58.750638 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.750433    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:13:58.750638 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.750529    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75fbfc4f44-6r7cr: secret "image-registry-tls" not found
Jul 07 02:13:58.750638 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:58.750582    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls podName:58d76295-5133-4a09-a23b-18ec6b3ed6fb nodeName:}" failed. No retries permitted until 2026-07-07 02:13:59.750566804 +0000 UTC m=+249.953987526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls") pod "image-registry-75fbfc4f44-6r7cr" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb") : secret "image-registry-tls" not found
Jul 07 02:13:59.202774 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.202679    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr" event={"ID":"3ff63f40-bce3-427c-990a-9545b6cc9d5b","Type":"ContainerStarted","Data":"bef0f3cdbb7be13a317a8bf7e21cf726a724347e27eefc1e02f46beb6f919cf5"}
Jul 07 02:13:59.204615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.204387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl" event={"ID":"09b78ba0-b49e-4412-b3e8-98d48570bb0a","Type":"ContainerStarted","Data":"9879dd7419e1e20edf5446ca4c43cb853cc8969bc4bf0c784c98ca3c5031c617"}
Jul 07 02:13:59.204615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.204459    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl" event={"ID":"09b78ba0-b49e-4412-b3e8-98d48570bb0a","Type":"ContainerStarted","Data":"fde49b522dc99eaa55a47359a9434bb6b1e0bd259d860354ed502e36ec40312d"}
Jul 07 02:13:59.205849 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.205809    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb" event={"ID":"a77aff70-34c1-4ab4-a4be-901ecebc8a7c","Type":"ContainerStarted","Data":"6ddd3d72837cdfc272e46719186ce0014233b08727b9bc4107d3f2126315186a"}
Jul 07 02:13:59.207344 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.207312    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" event={"ID":"45f4fec7-722a-4b66-98c1-185c65ad2816","Type":"ContainerStarted","Data":"58e1e4105265e4f4a205f37a38973933239bfb84f3317b53e6bf98f76d199642"}
Jul 07 02:13:59.223484 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.223221    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zr2kl" podStartSLOduration=1.223200977 podStartE2EDuration="1.223200977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:13:59.222786511 +0000 UTC m=+249.426207256" watchObservedRunningTime="2026-07-07 02:13:59.223200977 +0000 UTC m=+249.426621701"
Jul 07 02:13:59.758720 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.758679    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:13:59.758900 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:13:59.758777    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:13:59.758900 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:59.758862    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:13:59.758900 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:59.758886    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75fbfc4f44-6r7cr: secret "image-registry-tls" not found
Jul 07 02:13:59.758900 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:59.758889    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 02:13:59.759114 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:59.758959    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls podName:58d76295-5133-4a09-a23b-18ec6b3ed6fb nodeName:}" failed. No retries permitted until 2026-07-07 02:14:01.758936803 +0000 UTC m=+251.962357529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls") pod "image-registry-75fbfc4f44-6r7cr" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb") : secret "image-registry-tls" not found
Jul 07 02:13:59.759114 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:13:59.758978    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls podName:44a6ee7e-b1aa-4763-b5da-3e50450efa44 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:01.758969592 +0000 UTC m=+251.962390317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6b7pj" (UID: "44a6ee7e-b1aa-4763-b5da-3e50450efa44") : secret "samples-operator-tls" not found
Jul 07 02:14:00.692747 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:00.692711    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:00.747223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:00.747186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:01.777077 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:01.777051    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:14:01.777390 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:01.777112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:01.777390 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:01.777216    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:14:01.777390 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:01.777219    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 02:14:01.777390 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:01.777304    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls podName:44a6ee7e-b1aa-4763-b5da-3e50450efa44 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.777276887 +0000 UTC m=+255.980697614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6b7pj" (UID: "44a6ee7e-b1aa-4763-b5da-3e50450efa44") : secret "samples-operator-tls" not found
Jul 07 02:14:01.777390 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:01.777227    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75fbfc4f44-6r7cr: secret "image-registry-tls" not found
Jul 07 02:14:01.777390 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:01.777356    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls podName:58d76295-5133-4a09-a23b-18ec6b3ed6fb nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.777341428 +0000 UTC m=+255.980762150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls") pod "image-registry-75fbfc4f44-6r7cr" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb") : secret "image-registry-tls" not found
Jul 07 02:14:02.179385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.179346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:14:02.179571 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:02.179541    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 02:14:02.179631 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:02.179619    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs podName:478e76b1-c8b0-466d-aa7e-5dc477d27b41 nodeName:}" failed. No retries permitted until 2026-07-07 02:16:04.179598983 +0000 UTC m=+374.383019710 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs") pod "network-metrics-daemon-mzcc5" (UID: "478e76b1-c8b0-466d-aa7e-5dc477d27b41") : secret "metrics-daemon-secret" not found
Jul 07 02:14:02.218414 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.218382    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb" event={"ID":"a77aff70-34c1-4ab4-a4be-901ecebc8a7c","Type":"ContainerStarted","Data":"cb1fc5cc601540b3bcbe2abbb6803c1f0ed80ed2ee558823aa1d246e211acb85"}
Jul 07 02:14:02.220045 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.220026    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/0.log"
Jul 07 02:14:02.220162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.220060    2568 generic.go:356] "Generic (PLEG): container finished" podID="45f4fec7-722a-4b66-98c1-185c65ad2816" containerID="a74017496547e8a183fda93940572788f7c1c6a40b29b5ed00d70112fe4b2756" exitCode=255
Jul 07 02:14:02.220162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.220137    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" event={"ID":"45f4fec7-722a-4b66-98c1-185c65ad2816","Type":"ContainerDied","Data":"a74017496547e8a183fda93940572788f7c1c6a40b29b5ed00d70112fe4b2756"}
Jul 07 02:14:02.220335 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.220319    2568 scope.go:117] "RemoveContainer" containerID="a74017496547e8a183fda93940572788f7c1c6a40b29b5ed00d70112fe4b2756"
Jul 07 02:14:02.221629 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.221571    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr" event={"ID":"3ff63f40-bce3-427c-990a-9545b6cc9d5b","Type":"ContainerStarted","Data":"08e29ea16adf599baf32fe60767bf002d69452094cb11d17ef553382305fcb90"}
Jul 07 02:14:02.235786 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.235740    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb" podStartSLOduration=2.020597685 podStartE2EDuration="5.235721102s" totalImagesPullingTime="3.215123417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:02.235083176 +0000 UTC m=+252.438503956" watchObservedRunningTime="2026-07-07 02:14:02.235721102 +0000 UTC m=+252.439141847"
Jul 07 02:14:02.252228 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.252155    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr" podStartSLOduration=1.126543637 podStartE2EDuration="4.252138194s" totalImagesPullingTime="3.125594557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:02.252113215 +0000 UTC m=+252.455533961" watchObservedRunningTime="2026-07-07 02:14:02.252138194 +0000 UTC m=+252.455558939"
Jul 07 02:14:02.846398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.846352    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:02.937094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:02.937048    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.037632 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.037595    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.227754 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.227687    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:14:03.228187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.228168    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/0.log"
Jul 07 02:14:03.228288 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.228208    2568 generic.go:356] "Generic (PLEG): container finished" podID="45f4fec7-722a-4b66-98c1-185c65ad2816" containerID="89cd2111eda1ccc5247c9ed1d4cabc0c80330017689a76f218a877baf393dc02" exitCode=255
Jul 07 02:14:03.228339 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.228316    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" event={"ID":"45f4fec7-722a-4b66-98c1-185c65ad2816","Type":"ContainerDied","Data":"89cd2111eda1ccc5247c9ed1d4cabc0c80330017689a76f218a877baf393dc02"}
Jul 07 02:14:03.228387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.228371    2568 scope.go:117] "RemoveContainer" containerID="a74017496547e8a183fda93940572788f7c1c6a40b29b5ed00d70112fe4b2756"
Jul 07 02:14:03.228817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.228610    2568 scope.go:117] "RemoveContainer" containerID="89cd2111eda1ccc5247c9ed1d4cabc0c80330017689a76f218a877baf393dc02"
Jul 07 02:14:03.228917 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.228859    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-sdhnx_openshift-console-operator(45f4fec7-722a-4b66-98c1-185c65ad2816)\"" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" podUID="45f4fec7-722a-4b66-98c1-185c65ad2816"
Jul 07 02:14:03.282409 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.282380    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nh85c"]
Jul 07 02:14:03.285043 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.285016    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.287744 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.287711    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 02:14:03.287844 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.287717    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 02:14:03.287844 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.287810    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cczp7\""
Jul 07 02:14:03.298080 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.298054    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nh85c"]
Jul 07 02:14:03.319846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.319800    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.387194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.387158    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.390074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.390039    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7nhc\" (UniqueName: \"kubernetes.io/projected/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-api-access-g7nhc\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.390196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.390138    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-crio-socket\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.390235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.390208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.390271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.390235    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-data-volume\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.390443 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.390415    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.491950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.491921    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7nhc\" (UniqueName: \"kubernetes.io/projected/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-api-access-g7nhc\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.491963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-crio-socket\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.491992    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.492008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-data-volume\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.492039    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.492070    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.492131    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.492143    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:03.992127112 +0000 UTC m=+254.195547834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : configmap "kube-rbac-proxy" not found
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.492147    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-crio-socket\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.492199 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.492180    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:03.99216721 +0000 UTC m=+254.195587931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:03.492624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.492434    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-data-volume\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.503333 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.503309    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7nhc\" (UniqueName: \"kubernetes.io/projected/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-api-access-g7nhc\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.586850 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.586815    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.643174 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.643145    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.938749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.938649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:03.996763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.996728    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.996924 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:03.996780    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:03.996924 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.996853    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:14:03.996924 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.996898    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:03.997018 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.996931    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:04.996912323 +0000 UTC m=+255.200333056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : configmap "kube-rbac-proxy" not found
Jul 07 02:14:03.997018 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:03.996950    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:04.996942546 +0000 UTC m=+255.200363269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:04.039399 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:04.039366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:04.210083 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:04.210006    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:04.233155 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:04.233126    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:14:04.233510 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:04.233478    2568 scope.go:117] "RemoveContainer" containerID="89cd2111eda1ccc5247c9ed1d4cabc0c80330017689a76f218a877baf393dc02"
Jul 07 02:14:04.233674 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:04.233656    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-sdhnx_openshift-console-operator(45f4fec7-722a-4b66-98c1-185c65ad2816)\"" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" podUID="45f4fec7-722a-4b66-98c1-185c65ad2816"
Jul 07 02:14:04.274989 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:04.274955    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:05.005552 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.005510    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:05.006006 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.005579    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:05.006006 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.005608    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:14:05.006006 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.005671    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:07.005655883 +0000 UTC m=+257.209076605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : configmap "kube-rbac-proxy" not found
Jul 07 02:14:05.006006 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.005700    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:05.006006 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.005758    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:07.005740977 +0000 UTC m=+257.209161711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:05.693390 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.693352    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:05.797400 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.797363    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:05.807545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.807521    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-j2ck4"]
Jul 07 02:14:05.809677 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.809661    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:05.812254 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.812223    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:14:05.812402 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.812321    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:05.812402 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.812381    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 02:14:05.812519 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.812405    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 02:14:05.812519 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.812457    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:14:05.812519 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.812474    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75fbfc4f44-6r7cr: secret "image-registry-tls" not found
Jul 07 02:14:05.812519 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.812459    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls podName:44a6ee7e-b1aa-4763-b5da-3e50450efa44 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:13.812441009 +0000 UTC m=+264.015861743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6b7pj" (UID: "44a6ee7e-b1aa-4763-b5da-3e50450efa44") : secret "samples-operator-tls" not found
Jul 07 02:14:05.812743 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:05.812537    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls podName:58d76295-5133-4a09-a23b-18ec6b3ed6fb nodeName:}" failed. No retries permitted until 2026-07-07 02:14:13.812521277 +0000 UTC m=+264.015942019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls") pod "image-registry-75fbfc4f44-6r7cr" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb") : secret "image-registry-tls" not found
Jul 07 02:14:05.813667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.813645    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 02:14:05.813768 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.813670    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fvw2g\""
Jul 07 02:14:05.813768 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.813672    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 02:14:05.813863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.813655    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 02:14:05.820460 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.819324    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-j2ck4"]
Jul 07 02:14:05.853658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.853624    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:05.912796 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.912757    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-signing-key\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:05.912796 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.912793    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7fxp\" (UniqueName: \"kubernetes.io/projected/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-kube-api-access-d7fxp\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:05.913007 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:05.912816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-signing-cabundle\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.014095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.014058    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-signing-key\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.014562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.014099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7fxp\" (UniqueName: \"kubernetes.io/projected/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-kube-api-access-d7fxp\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.014562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.014124    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-signing-cabundle\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.014888 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.014866    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-signing-cabundle\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.016555 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.016535    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-signing-key\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.024415 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.024389    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7fxp\" (UniqueName: \"kubernetes.io/projected/89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b-kube-api-access-d7fxp\") pod \"service-ca-758f599c5-j2ck4\" (UID: \"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b\") " pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.073827 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.073799    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2wnp_b6144ddf-5525-45aa-afb7-ef69d27cab03/dns-node-resolver/0.log"
Jul 07 02:14:06.123250 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.123211    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-j2ck4"
Jul 07 02:14:06.246326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.246288    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-j2ck4"]
Jul 07 02:14:06.250635 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:06.250607    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b69a4e_5c3d_4b8a_9e20_2dbc2a2a304b.slice/crio-ad6672f5496897fdf5bf14fc297b5351acad4224105d4d92d5382b2e971ea14a WatchSource:0}: Error finding container ad6672f5496897fdf5bf14fc297b5351acad4224105d4d92d5382b2e971ea14a: Status 404 returned error can't find the container with id ad6672f5496897fdf5bf14fc297b5351acad4224105d4d92d5382b2e971ea14a
Jul 07 02:14:06.770372 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:06.770337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:07.023476 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.023384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:07.023476 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.023452    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:07.023915 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:07.023524    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:14:07.023915 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:07.023595    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:11.023575593 +0000 UTC m=+261.226996316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : configmap "kube-rbac-proxy" not found
Jul 07 02:14:07.023915 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:07.023629    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:07.023915 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:07.023675    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:11.023663517 +0000 UTC m=+261.227084239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:07.073911 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.073879    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r7s9h_d500a857-2643-4108-80af-8da289f0f3c7/node-ca/0.log"
Jul 07 02:14:07.244504 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.244455    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-j2ck4" event={"ID":"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b","Type":"ContainerStarted","Data":"2c08d7974991866aea0efa20b37fb0004f881de344bcbb85cfdae07c8a093f00"}
Jul 07 02:14:07.244676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.244524    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-j2ck4" event={"ID":"89b69a4e-5c3d-4b8a-9e20-2dbc2a2a304b","Type":"ContainerStarted","Data":"ad6672f5496897fdf5bf14fc297b5351acad4224105d4d92d5382b2e971ea14a"}
Jul 07 02:14:07.262745 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.262674    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-j2ck4" podStartSLOduration=2.262653844 podStartE2EDuration="2.262653844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:07.261063081 +0000 UTC m=+257.464483825" watchObservedRunningTime="2026-07-07 02:14:07.262653844 +0000 UTC m=+257.466074590"
Jul 07 02:14:07.640862 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.640829    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:07.828094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.827953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:07.936062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:07.935960    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:08.037762 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.037728    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:08.078178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.078142    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xt4rs_a387479e-f772-49bc-b157-c146c9358d3e/serve-healthcheck-canary/0.log"
Jul 07 02:14:08.138223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.138147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:08.199100 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.198986    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:08.245000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.244970    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:14:08.245182 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.245022    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:14:08.245490 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.245462    2568 scope.go:117] "RemoveContainer" containerID="89cd2111eda1ccc5247c9ed1d4cabc0c80330017689a76f218a877baf393dc02"
Jul 07 02:14:08.245688 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:08.245657    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-sdhnx_openshift-console-operator(45f4fec7-722a-4b66-98c1-185c65ad2816)\"" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" podUID="45f4fec7-722a-4b66-98c1-185c65ad2816"
Jul 07 02:14:08.344552 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:08.344519    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:09.030120 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:09.030081    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:09.131309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:09.131269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:09.337475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:09.337375    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:09.749134 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:09.749103    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:09.943919 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:09.943886    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:10.044754 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:10.044663    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:10.145363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:10.145327    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:10.545902 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:10.545870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:10.942891 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:10.942799    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:11.057303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:11.057264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:11.057463 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:11.057318    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:11.057463 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:11.057400    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:14:11.057463 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:11.057442    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:11.057597 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:11.057472    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:19.057454597 +0000 UTC m=+269.260875339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : configmap "kube-rbac-proxy" not found
Jul 07 02:14:11.057597 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:11.057490    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls podName:d60bc751-eec6-4ff7-9a52-aad22cfeb3ce nodeName:}" failed. No retries permitted until 2026-07-07 02:14:19.057484254 +0000 UTC m=+269.260904975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nh85c" (UID: "d60bc751-eec6-4ff7-9a52-aad22cfeb3ce") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:14:11.536315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:11.536225    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:11.637351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:11.637314    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:11.696674 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:11.696645    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:12.142933 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:12.142902    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:12.334536 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:12.334481    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:12.737490 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:12.737440    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:12.936570 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:12.936538    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:13.356961 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.356930    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:13.885028 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.884994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:14:13.885418 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.885066    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:13.888049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.888011    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"image-registry-75fbfc4f44-6r7cr\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") " pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:13.888168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.888011    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44a6ee7e-b1aa-4763-b5da-3e50450efa44-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6b7pj\" (UID: \"44a6ee7e-b1aa-4763-b5da-3e50450efa44\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:14:13.942923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.942881    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:13.952725 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.952703    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"
Jul 07 02:14:13.956648 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:13.956633    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:14.099422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.099391    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75fbfc4f44-6r7cr"]
Jul 07 02:14:14.102019 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:14.101988    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d76295_5133_4a09_a23b_18ec6b3ed6fb.slice/crio-0a3e0255554b6e57680bd537b9a6f8ac3f92f66764a14498e11773e495296388 WatchSource:0}: Error finding container 0a3e0255554b6e57680bd537b9a6f8ac3f92f66764a14498e11773e495296388: Status 404 returned error can't find the container with id 0a3e0255554b6e57680bd537b9a6f8ac3f92f66764a14498e11773e495296388
Jul 07 02:14:14.111259 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.111235    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj"]
Jul 07 02:14:14.142977 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.142940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:14.268029 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.267990    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj" event={"ID":"44a6ee7e-b1aa-4763-b5da-3e50450efa44","Type":"ContainerStarted","Data":"a67ca0be1ea4bab12c77c6ed84c30a026d508e1e3f9af8b53abd0ca96f79172a"}
Jul 07 02:14:14.269571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.269541    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr" event={"ID":"58d76295-5133-4a09-a23b-18ec6b3ed6fb","Type":"ContainerStarted","Data":"2c5a6abc37074ad49f26ad9a826c5b4c64c9adb743202e7c1c895ee3d832f91b"}
Jul 07 02:14:14.269708 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.269590    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr" event={"ID":"58d76295-5133-4a09-a23b-18ec6b3ed6fb","Type":"ContainerStarted","Data":"0a3e0255554b6e57680bd537b9a6f8ac3f92f66764a14498e11773e495296388"}
Jul 07 02:14:14.269708 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.269657    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:14.289178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.289124    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr" podStartSLOduration=16.289104743 podStartE2EDuration="16.289104743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:14.288100235 +0000 UTC m=+264.491520979" watchObservedRunningTime="2026-07-07 02:14:14.289104743 +0000 UTC m=+264.492525489"
Jul 07 02:14:14.339762 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.339729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:14.541422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.541386    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:14.737605 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:14.737558    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:15.736553 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:15.736516    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:16.028003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.027966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:16.135754 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.135723    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:16.279040 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.278961    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj" event={"ID":"44a6ee7e-b1aa-4763-b5da-3e50450efa44","Type":"ContainerStarted","Data":"bc1a317f303276a33a55dcef44ccc8ba54e8faf701dda13542e5918e260cc589"}
Jul 07 02:14:16.279040 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.279019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj" event={"ID":"44a6ee7e-b1aa-4763-b5da-3e50450efa44","Type":"ContainerStarted","Data":"3e8f88f5f9fa567a5b9d2925f734fa0182d07750d9f057caa89db7cb275a9343"}
Jul 07 02:14:16.306038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.305989    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6b7pj" podStartSLOduration=16.379793335 podStartE2EDuration="18.30597472s" totalImagesPullingTime="1.926181385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:16.305186473 +0000 UTC m=+266.508607217" watchObservedRunningTime="2026-07-07 02:14:16.30597472 +0000 UTC m=+266.509395464"
Jul 07 02:14:16.365607 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.365577    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:16.534856 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.534756    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:16.740771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:16.740737    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:17.745693 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:17.745657    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:17.934857 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:17.934826    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:18.142556 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:18.142447    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:18.337425 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:18.337386    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:18.437696 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:18.437604    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:19.131802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:19.131761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:19.132273 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:19.131815    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:19.132368 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:19.132344    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:19.134223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:19.134201    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d60bc751-eec6-4ff7-9a52-aad22cfeb3ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nh85c\" (UID: \"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce\") " pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:19.194574 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:19.194542    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nh85c"
Jul 07 02:14:19.319757 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:19.319734    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nh85c"]
Jul 07 02:14:19.322290 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:19.322255    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60bc751_eec6_4ff7_9a52_aad22cfeb3ce.slice/crio-96883f40f50aa4172611714ddeda729fc1b25cb42cb849f082fd37cc22f65609 WatchSource:0}: Error finding container 96883f40f50aa4172611714ddeda729fc1b25cb42cb849f082fd37cc22f65609: Status 404 returned error can't find the container with id 96883f40f50aa4172611714ddeda729fc1b25cb42cb849f082fd37cc22f65609
Jul 07 02:14:20.293393 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:20.293313    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nh85c" event={"ID":"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce","Type":"ContainerStarted","Data":"96883f40f50aa4172611714ddeda729fc1b25cb42cb849f082fd37cc22f65609"}
Jul 07 02:14:20.429588 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:20.429551    2568 scope.go:117] "RemoveContainer" containerID="89cd2111eda1ccc5247c9ed1d4cabc0c80330017689a76f218a877baf393dc02"
Jul 07 02:14:21.298827 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:21.298798    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:14:21.299282 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:21.298901    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" event={"ID":"45f4fec7-722a-4b66-98c1-185c65ad2816","Type":"ContainerStarted","Data":"6285b10b2e6efa10bf19058db868a6cd525985080d8cee6f08243307d39d51f8"}
Jul 07 02:14:21.299439 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:21.299393    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:14:21.315825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:21.315755    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx" podStartSLOduration=21.083353232 podStartE2EDuration="24.315734887s" totalImagesPullingTime="3.232381655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:21.315454169 +0000 UTC m=+271.518874914" watchObservedRunningTime="2026-07-07 02:14:21.315734887 +0000 UTC m=+271.519155633"
Jul 07 02:14:22.180702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:22.180669    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-sdhnx"
Jul 07 02:14:22.348753 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:22.348726    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:22.449079 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:22.448952    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:22.507435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:22.507390    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:22.653029 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:22.652991    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:23.032426 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:23.032382    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:23.132580 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:23.132479    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:23.439472 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:23.439442    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:24.048714 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.048680    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:24.111148 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.111115    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:24.283656 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.283613    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:24.311855 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.311828    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nh85c" event={"ID":"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce","Type":"ContainerStarted","Data":"fc84a58bde16b96d803c1b397e28edc94d45a3237f831c74a313a3a542132937"}
Jul 07 02:14:24.311997 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.311883    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nh85c" event={"ID":"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce","Type":"ContainerStarted","Data":"ee88fd066e8935042d616deb3c0bc6fd8b76912fe718936c52d26273f22c0019"}
Jul 07 02:14:24.338439 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.338362    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:24.656142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:24.656034    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:26.498592 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.498557    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-vk9xl"]
Jul 07 02:14:26.500974 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.500955    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75fbfc4f44-6r7cr"]
Jul 07 02:14:26.501113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.501089    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:26.504507 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.504471    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rmk7m\""
Jul 07 02:14:26.504621 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.504605    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 02:14:26.504737 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.504721    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 02:14:26.516456 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.516437    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-vk9xl"]
Jul 07 02:14:26.594375 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.594336    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppbks\" (UniqueName: \"kubernetes.io/projected/1c8724da-6d8a-40d9-b1f1-821764073753-kube-api-access-ppbks\") pod \"downloads-685bfb5f64-vk9xl\" (UID: \"1c8724da-6d8a-40d9-b1f1-821764073753\") " pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:26.615421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.615386    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:26.694771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.694695    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppbks\" (UniqueName: \"kubernetes.io/projected/1c8724da-6d8a-40d9-b1f1-821764073753-kube-api-access-ppbks\") pod \"downloads-685bfb5f64-vk9xl\" (UID: \"1c8724da-6d8a-40d9-b1f1-821764073753\") " pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:26.706009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.705975    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:26.712732 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.712714    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppbks\" (UniqueName: \"kubernetes.io/projected/1c8724da-6d8a-40d9-b1f1-821764073753-kube-api-access-ppbks\") pod \"downloads-685bfb5f64-vk9xl\" (UID: \"1c8724da-6d8a-40d9-b1f1-821764073753\") " pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:26.778964 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.778931    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:26.810488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.810446    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:26.926449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.926419    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:26.956755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:26.956678    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-vk9xl"]
Jul 07 02:14:26.959372 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:26.959346    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8724da_6d8a_40d9_b1f1_821764073753.slice/crio-561b69d6335a4f2161ce5b8e4b3c435a907848b8d55740e2e24b7bc237e436e9 WatchSource:0}: Error finding container 561b69d6335a4f2161ce5b8e4b3c435a907848b8d55740e2e24b7bc237e436e9: Status 404 returned error can't find the container with id 561b69d6335a4f2161ce5b8e4b3c435a907848b8d55740e2e24b7bc237e436e9
Jul 07 02:14:27.323126 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:27.323092    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nh85c" event={"ID":"d60bc751-eec6-4ff7-9a52-aad22cfeb3ce","Type":"ContainerStarted","Data":"5cf69fcdaa28fb282e6a821c345b546f05f71324c6a1e3c8e67b9afe94369cad"}
Jul 07 02:14:27.324191 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:27.324165    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-vk9xl" event={"ID":"1c8724da-6d8a-40d9-b1f1-821764073753","Type":"ContainerStarted","Data":"561b69d6335a4f2161ce5b8e4b3c435a907848b8d55740e2e24b7bc237e436e9"}
Jul 07 02:14:27.339846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:27.339799    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nh85c" podStartSLOduration=17.379598411 podStartE2EDuration="24.339781968s" totalImagesPullingTime="6.960183557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:14:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:27.338757514 +0000 UTC m=+277.542178259" watchObservedRunningTime="2026-07-07 02:14:27.339781968 +0000 UTC m=+277.543202713"
Jul 07 02:14:30.565543 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:30.565506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:30.663366 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:30.663329    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:30.995798 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:30.995760    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9zcvr" podUID="a5a8d96c-2c84-479e-a76c-de376aca0782"
Jul 07 02:14:31.340453 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:31.340381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:14:33.012769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:33.012730    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:34.179250 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:34.179218    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:34.967641 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:34.967603    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:14:34.970206 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:34.970177    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a8d96c-2c84-479e-a76c-de376aca0782-metrics-tls\") pod \"dns-default-9zcvr\" (UID: \"a5a8d96c-2c84-479e-a76c-de376aca0782\") " pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:14:35.244049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:35.244011    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tv6md\""
Jul 07 02:14:35.252178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:35.252143    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:14:35.403235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:35.403156    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9zcvr"]
Jul 07 02:14:35.405847 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:35.405817    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a8d96c_2c84_479e_a76c_de376aca0782.slice/crio-9cc03e7013975b0d11e2f8f2fcecf6af3fc0b9be06fec125f80e52d0603dc7ba WatchSource:0}: Error finding container 9cc03e7013975b0d11e2f8f2fcecf6af3fc0b9be06fec125f80e52d0603dc7ba: Status 404 returned error can't find the container with id 9cc03e7013975b0d11e2f8f2fcecf6af3fc0b9be06fec125f80e52d0603dc7ba
Jul 07 02:14:36.195641 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:36.195604    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:36.262645 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:36.262598    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:36.361171 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:36.361127    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zcvr" event={"ID":"a5a8d96c-2c84-479e-a76c-de376aca0782","Type":"ContainerStarted","Data":"9cc03e7013975b0d11e2f8f2fcecf6af3fc0b9be06fec125f80e52d0603dc7ba"}
Jul 07 02:14:36.508539 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:36.508456    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:36.536393 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:36.536305    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:37.278070 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.278032    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79b66c9fc6-fd545"]
Jul 07 02:14:37.282568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.282539    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.285182 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.285155    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 02:14:37.285336 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.285215    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 02:14:37.285466 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.285448    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 02:14:37.286537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.286313    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 02:14:37.286537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.286390    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 02:14:37.286537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.286455    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cnxzk\""
Jul 07 02:14:37.293667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.293645    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79b66c9fc6-fd545"]
Jul 07 02:14:37.368178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.368144    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zcvr" event={"ID":"a5a8d96c-2c84-479e-a76c-de376aca0782","Type":"ContainerStarted","Data":"e30a72fac63df8d9437f05b101d6f14dca41d85193a4393c8d337a3aaf924d67"}
Jul 07 02:14:37.389285 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.389245    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-oauth-serving-cert\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.389472 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.389319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-console-config\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.389472 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.389376    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-oauth-config\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.389472 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.389441    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp78s\" (UniqueName: \"kubernetes.io/projected/881889e3-fc82-4d70-9c62-2619740e2615-kube-api-access-pp78s\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.389830 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.389509    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-service-ca\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.389830 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.389555    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-serving-cert\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.490197 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.490160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp78s\" (UniqueName: \"kubernetes.io/projected/881889e3-fc82-4d70-9c62-2619740e2615-kube-api-access-pp78s\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.490590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.490212    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-service-ca\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.490590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.490240    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-serving-cert\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.490590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.490261    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-oauth-serving-cert\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.490590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.490285    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-console-config\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.490590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.490304    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-oauth-config\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.491266 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.491110    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-service-ca\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.491266 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.491188    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-console-config\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.491266 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.491212    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-oauth-serving-cert\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.493382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.493361    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-serving-cert\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.493478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.493462    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-oauth-config\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.498694 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.498667    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp78s\" (UniqueName: \"kubernetes.io/projected/881889e3-fc82-4d70-9c62-2619740e2615-kube-api-access-pp78s\") pod \"console-79b66c9fc6-fd545\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") " pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.595737 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.595698    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:37.746704 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:37.746617    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79b66c9fc6-fd545"]
Jul 07 02:14:37.750715 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:37.750681    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881889e3_fc82_4d70_9c62_2619740e2615.slice/crio-c7b6b45aac790c2d1106816eb2238e0ae666b0a6ca5a236377090e6f6fcd3918 WatchSource:0}: Error finding container c7b6b45aac790c2d1106816eb2238e0ae666b0a6ca5a236377090e6f6fcd3918: Status 404 returned error can't find the container with id c7b6b45aac790c2d1106816eb2238e0ae666b0a6ca5a236377090e6f6fcd3918
Jul 07 02:14:38.374874 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:38.374842    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zcvr" event={"ID":"a5a8d96c-2c84-479e-a76c-de376aca0782","Type":"ContainerStarted","Data":"75bcad0fd8d081d14243cd0d2ae06f082f983fb4cd3e669ab61672bcb5bb67fe"}
Jul 07 02:14:38.375345 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:38.374929    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:14:38.376315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:38.376273    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b66c9fc6-fd545" event={"ID":"881889e3-fc82-4d70-9c62-2619740e2615","Type":"ContainerStarted","Data":"c7b6b45aac790c2d1106816eb2238e0ae666b0a6ca5a236377090e6f6fcd3918"}
Jul 07 02:14:38.399300 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:38.399232    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9zcvr" podStartSLOduration=252.572430366 podStartE2EDuration="4m14.39921215s" totalImagesPullingTime="1.826781784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:38.39880425 +0000 UTC m=+288.602224995" watchObservedRunningTime="2026-07-07 02:14:38.39921215 +0000 UTC m=+288.602632897"
Jul 07 02:14:38.807968 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:38.807927    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:38.908739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:38.908646    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:39.033366 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.033322    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:39.285157 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.285124    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689b584fdc-9dfx9"]
Jul 07 02:14:39.290825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.290795    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.292445 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.292418    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:39.299645 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.299616    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689b584fdc-9dfx9"]
Jul 07 02:14:39.300740 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.300718    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 02:14:39.351951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.351910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:39.409511 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409461    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbbk5\" (UniqueName: \"kubernetes.io/projected/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-kube-api-access-zbbk5\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.409965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409530    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-oauth-config\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.409965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-trusted-ca-bundle\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.409965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409620    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-serving-cert\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.409965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409733    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-oauth-serving-cert\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.409965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-config\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.409965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.409822    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-service-ca\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.465317 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.465178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:39.511294 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511257    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-serving-cert\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.511489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511340    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-oauth-serving-cert\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.511489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-config\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.511489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-service-ca\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.511489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbbk5\" (UniqueName: \"kubernetes.io/projected/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-kube-api-access-zbbk5\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.511730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511515    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-oauth-config\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.511730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.511547    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-trusted-ca-bundle\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.512615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.512572    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-service-ca\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.512851 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.512828    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-trusted-ca-bundle\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.512948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.512864    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-oauth-serving-cert\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.512948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.512921    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-config\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.515262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.515239    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-oauth-config\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.515633 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.515607    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-serving-cert\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.518913 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.518839    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:39.521277 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.521256    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbbk5\" (UniqueName: \"kubernetes.io/projected/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-kube-api-access-zbbk5\") pod \"console-689b584fdc-9dfx9\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") " pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:39.604445 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:39.604359    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:40.791013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:40.790972    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:40.846284 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:40.846242    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:41.326814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.326775    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:41.539654 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.539618    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:41.579750 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.579660    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kw7tf"]
Jul 07 02:14:41.583370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.583349    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.586051 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.586029    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m6bjf\""
Jul 07 02:14:41.588090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.587609    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 02:14:41.588090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.587735    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 02:14:41.588090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.587810    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 02:14:41.588090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.587841    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 02:14:41.588823 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.588803    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 02:14:41.589057 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.589040    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 02:14:41.638742 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.638667    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:41.729977 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.729940    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-wtmp\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730185 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8fbc0dc-24ac-4580-8c92-52773eb35827-metrics-client-ca\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730185 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730129    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730185 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730159    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-root\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730207    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-accelerators-collector-config\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730234    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-sys\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730304    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-tls\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730331    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcnjm\" (UniqueName: \"kubernetes.io/projected/e8fbc0dc-24ac-4580-8c92-52773eb35827-kube-api-access-tcnjm\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.730585 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.730359    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-textfile\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.738654 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.738615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:41.831119 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-sys\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831119 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831107    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-tls\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831135    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcnjm\" (UniqueName: \"kubernetes.io/projected/e8fbc0dc-24ac-4580-8c92-52773eb35827-kube-api-access-tcnjm\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831161    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-textfile\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831165    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-sys\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831204    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-wtmp\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831241    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8fbc0dc-24ac-4580-8c92-52773eb35827-metrics-client-ca\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831315    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-root\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.831658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-accelerators-collector-config\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.832055 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.831986    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-accelerators-collector-config\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.832104 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.832053    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-root\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.832196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.832176    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-wtmp\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.832295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.832268    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-textfile\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.832703 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.832662    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8fbc0dc-24ac-4580-8c92-52773eb35827-metrics-client-ca\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.834869 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.834847    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.834989 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.834878    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8fbc0dc-24ac-4580-8c92-52773eb35827-node-exporter-tls\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.839241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.839164    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcnjm\" (UniqueName: \"kubernetes.io/projected/e8fbc0dc-24ac-4580-8c92-52773eb35827-kube-api-access-tcnjm\") pod \"node-exporter-kw7tf\" (UID: \"e8fbc0dc-24ac-4580-8c92-52773eb35827\") " pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.895529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.895437    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:41.896281 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.896255    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kw7tf"
Jul 07 02:14:41.994426 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:41.994350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:42.093608 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.093520    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:42.194192 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.194152    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:42.573440 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.573396    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:42.666756 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.666721    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:14:42.670340 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.670308    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.673079 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.672922    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 02:14:42.673079 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.672990    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 02:14:42.673309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673230    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 02:14:42.673309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673271    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 02:14:42.673571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673450    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 02:14:42.673571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:42.673692 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673665    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 02:14:42.673748 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673732    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 02:14:42.673864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673768    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r9qdg\""
Jul 07 02:14:42.673864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673801    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 02:14:42.673864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.673857    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 02:14:42.691068 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.691040    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:14:42.740210 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740170    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740392 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740222    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-web-config\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740392 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740261    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740392 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44hgt\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-kube-api-access-44hgt\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740392 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740355    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740392 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740377    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740457    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740513    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740567    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-out\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.740925 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.740703    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842060    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842109    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-web-config\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842166    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44hgt\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-kube-api-access-44hgt\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842187    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842207    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842238    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842261    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.842307    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:42.842397    2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 07 02:14:42.842730 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:42.842451    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls podName:7d23ab93-ef4c-4961-a780-c036ed541bbe nodeName:}" failed. No retries permitted until 2026-07-07 02:14:43.342432822 +0000 UTC m=+293.545853548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe") : secret "alertmanager-main-tls" not found
Jul 07 02:14:42.843546 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.843177    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.843546 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.843208    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-out\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.843546 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.843253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.843546 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.843287    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.843546 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.843341    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.845147 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.844597    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.845147 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.845051    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.846096 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.846068    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.846308 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.846284    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.847060 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.847035    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-web-config\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.847884 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.847857    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.848258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.848214    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.848258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.848236    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.848406 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.848339    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-out\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.848406 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.848350    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:42.858196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:42.858148    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44hgt\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-kube-api-access-44hgt\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:43.349245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.349212    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:43.349423 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:43.349393    2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 07 02:14:43.349511 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:43.349462    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls podName:7d23ab93-ef4c-4961-a780-c036ed541bbe nodeName:}" failed. No retries permitted until 2026-07-07 02:14:44.349444609 +0000 UTC m=+294.552865332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe") : secret "alertmanager-main-tls" not found
Jul 07 02:14:43.568676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.568638    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b4d79695c-592zr"]
Jul 07 02:14:43.574416 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.574386    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.576972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.576941    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 07 02:14:43.577118 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.577038    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 07 02:14:43.577184 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.577164    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 07 02:14:43.578110 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.578083    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fbffledp2e46h\""
Jul 07 02:14:43.578110 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.578105    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 07 02:14:43.578262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.578149    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 07 02:14:43.578262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.578191    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8c8bm\""
Jul 07 02:14:43.583333 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.583187    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b4d79695c-592zr"]
Jul 07 02:14:43.631927 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.631847    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:43.652151 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652113    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c1e295e-29cc-4033-be75-609fffee5be9-metrics-client-ca\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-grpc-tls\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652226    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652269    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652362    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-tls\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652391    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz9hh\" (UniqueName: \"kubernetes.io/projected/6c1e295e-29cc-4033-be75-609fffee5be9-kube-api-access-rz9hh\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652484    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.652696 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.652560    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.753776 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.753730    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c1e295e-29cc-4033-be75-609fffee5be9-metrics-client-ca\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.753776 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.753779    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-grpc-tls\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754026 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.753896    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754026 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.753940    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754026 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.753994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-tls\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754026 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.754016    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz9hh\" (UniqueName: \"kubernetes.io/projected/6c1e295e-29cc-4033-be75-609fffee5be9-kube-api-access-rz9hh\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754211 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.754087    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754211 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.754155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.754680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.754595    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c1e295e-29cc-4033-be75-609fffee5be9-metrics-client-ca\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.757042 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.757022    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.757165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.757148    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-grpc-tls\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.757255 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.757235    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.757578 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.757532    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.757578 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.757548    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-tls\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.757899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.757876    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c1e295e-29cc-4033-be75-609fffee5be9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.768158 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.768135    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz9hh\" (UniqueName: \"kubernetes.io/projected/6c1e295e-29cc-4033-be75-609fffee5be9-kube-api-access-rz9hh\") pod \"thanos-querier-5b4d79695c-592zr\" (UID: \"6c1e295e-29cc-4033-be75-609fffee5be9\") " pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:43.886653 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:43.886554    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:44.360775 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.360740    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:44.363353 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.363326    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:44.483625 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.483571    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:14:44.522856 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.522823    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:44.623703 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.623625    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:44.723988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.723951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:44.795066 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:44.795030    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:45.075189 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.075127    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:45.175326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.175269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:45.983223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.983192    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5cbfc84df-p225k"]
Jul 07 02:14:45.986448 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.986426    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:45.990724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.990582    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 02:14:45.990724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.990597    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 02:14:45.990724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.990591    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f0omrg2o8ipj9\""
Jul 07 02:14:45.990724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.990641    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xdp2t\""
Jul 07 02:14:45.990724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.990597    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 02:14:45.991038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.990582    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 02:14:45.997543 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:45.997519    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5cbfc84df-p225k"]
Jul 07 02:14:46.077622 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077591    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-secret-metrics-server-client-certs\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.077622 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077627    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5aa3034f-8e93-4e74-907c-be6fa9bc1085-metrics-server-audit-profiles\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.078095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5aa3034f-8e93-4e74-907c-be6fa9bc1085-audit-log\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.078095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-secret-metrics-server-tls\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.078095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077820    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llst5\" (UniqueName: \"kubernetes.io/projected/5aa3034f-8e93-4e74-907c-be6fa9bc1085-kube-api-access-llst5\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.078095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077843    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa3034f-8e93-4e74-907c-be6fa9bc1085-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.078095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.077889    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-client-ca-bundle\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179275 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179237    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llst5\" (UniqueName: \"kubernetes.io/projected/5aa3034f-8e93-4e74-907c-be6fa9bc1085-kube-api-access-llst5\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179275 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa3034f-8e93-4e74-907c-be6fa9bc1085-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179332    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-client-ca-bundle\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179410    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-secret-metrics-server-client-certs\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179438    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5aa3034f-8e93-4e74-907c-be6fa9bc1085-metrics-server-audit-profiles\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5aa3034f-8e93-4e74-907c-be6fa9bc1085-audit-log\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.179769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.179579    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-secret-metrics-server-tls\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.180190 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.180137    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa3034f-8e93-4e74-907c-be6fa9bc1085-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.180545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.180480    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5aa3034f-8e93-4e74-907c-be6fa9bc1085-metrics-server-audit-profiles\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.180651 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.180574    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5aa3034f-8e93-4e74-907c-be6fa9bc1085-audit-log\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.182326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.182304    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-secret-metrics-server-client-certs\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.182670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.182650    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-client-ca-bundle\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.182746 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.182718    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5aa3034f-8e93-4e74-907c-be6fa9bc1085-secret-metrics-server-tls\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.187322 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.187299    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llst5\" (UniqueName: \"kubernetes.io/projected/5aa3034f-8e93-4e74-907c-be6fa9bc1085-kube-api-access-llst5\") pod \"metrics-server-5cbfc84df-p225k\" (UID: \"5aa3034f-8e93-4e74-907c-be6fa9bc1085\") " pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.298519 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.298455    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:14:46.362720 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.362689    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"]
Jul 07 02:14:46.368814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.368791    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.368955 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.368792    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79b66c9fc6-fd545"]
Jul 07 02:14:46.371177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.371154    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 02:14:46.371322 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.371277    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vgk4l\""
Jul 07 02:14:46.374046 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.373722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:46.374924 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.374901    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"]
Jul 07 02:14:46.427858 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:14:46.427816    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mzcc5" podUID="478e76b1-c8b0-466d-aa7e-5dc477d27b41"
Jul 07 02:14:46.465391 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.465350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:46.482123 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.482094    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd0163-83d3-415b-a97b-ea13e64a2afb-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-89j5z\" (UID: \"4dfd0163-83d3-415b-a97b-ea13e64a2afb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.482296 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.482187    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97szw\" (UniqueName: \"kubernetes.io/projected/4dfd0163-83d3-415b-a97b-ea13e64a2afb-kube-api-access-97szw\") pod \"monitoring-plugin-d5dcff7cf-89j5z\" (UID: \"4dfd0163-83d3-415b-a97b-ea13e64a2afb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.583170 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.583089    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd0163-83d3-415b-a97b-ea13e64a2afb-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-89j5z\" (UID: \"4dfd0163-83d3-415b-a97b-ea13e64a2afb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.583324 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.583191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97szw\" (UniqueName: \"kubernetes.io/projected/4dfd0163-83d3-415b-a97b-ea13e64a2afb-kube-api-access-97szw\") pod \"monitoring-plugin-d5dcff7cf-89j5z\" (UID: \"4dfd0163-83d3-415b-a97b-ea13e64a2afb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.585915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.585882    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd0163-83d3-415b-a97b-ea13e64a2afb-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-89j5z\" (UID: \"4dfd0163-83d3-415b-a97b-ea13e64a2afb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.599950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.599922    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97szw\" (UniqueName: \"kubernetes.io/projected/4dfd0163-83d3-415b-a97b-ea13e64a2afb-kube-api-access-97szw\") pod \"monitoring-plugin-d5dcff7cf-89j5z\" (UID: \"4dfd0163-83d3-415b-a97b-ea13e64a2afb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:46.681791 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:46.681755    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:47.218061 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:47.218023    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fbc0dc_24ac_4580_8c92_52773eb35827.slice/crio-8093b326ff1c5e29e76efe7c7519c3d26e88c53fcfe42153a4b7ef32ca34d032 WatchSource:0}: Error finding container 8093b326ff1c5e29e76efe7c7519c3d26e88c53fcfe42153a4b7ef32ca34d032: Status 404 returned error can't find the container with id 8093b326ff1c5e29e76efe7c7519c3d26e88c53fcfe42153a4b7ef32ca34d032
Jul 07 02:14:47.400018 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.399781    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"]
Jul 07 02:14:47.413951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.413917    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-vk9xl" event={"ID":"1c8724da-6d8a-40d9-b1f1-821764073753","Type":"ContainerStarted","Data":"ecca2ecc6d8e09dac889fd86ca7bd99604c23d0473893d3dc1904ac8e4b978c8"}
Jul 07 02:14:47.421814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.416371    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689b584fdc-9dfx9"]
Jul 07 02:14:47.421814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.418126    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b66c9fc6-fd545" event={"ID":"881889e3-fc82-4d70-9c62-2619740e2615","Type":"ContainerStarted","Data":"c1bc6de6556486d097093b13c06dd3f84fa7e51cd84458ba33ca7442f2ba3685"}
Jul 07 02:14:47.423077 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.423019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z" event={"ID":"4dfd0163-83d3-415b-a97b-ea13e64a2afb","Type":"ContainerStarted","Data":"bb04ca7e2011c1a06dffa8eccd1fe995aa1267160da391db9b47a4b455b76270"}
Jul 07 02:14:47.425409 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.425336    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kw7tf" event={"ID":"e8fbc0dc-24ac-4580-8c92-52773eb35827","Type":"ContainerStarted","Data":"8093b326ff1c5e29e76efe7c7519c3d26e88c53fcfe42153a4b7ef32ca34d032"}
Jul 07 02:14:47.427899 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:47.427876    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81a3ac3_0df3_404b_ad9f_d4f4c49b2c60.slice/crio-2e5af1f88b8e2cc50ff07698a060fcab07b81e25397ae19bae02427629cf1e07 WatchSource:0}: Error finding container 2e5af1f88b8e2cc50ff07698a060fcab07b81e25397ae19bae02427629cf1e07: Status 404 returned error can't find the container with id 2e5af1f88b8e2cc50ff07698a060fcab07b81e25397ae19bae02427629cf1e07
Jul 07 02:14:47.435927 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.435868    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79b66c9fc6-fd545" podStartSLOduration=0.971827164 podStartE2EDuration="10.435852758s" totalImagesPullingTime="9.464025594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:47.435062288 +0000 UTC m=+297.638483035" watchObservedRunningTime="2026-07-07 02:14:47.435852758 +0000 UTC m=+297.639273501"
Jul 07 02:14:47.596731 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.596694    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:14:47.658037 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.658014    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5cbfc84df-p225k"]
Jul 07 02:14:47.660813 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:47.660784    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa3034f_8e93_4e74_907c_be6fa9bc1085.slice/crio-b29229c861de515f82f2b6a9af7ee906b0efb0900ea7417cbe71925af01b568f WatchSource:0}: Error finding container b29229c861de515f82f2b6a9af7ee906b0efb0900ea7417cbe71925af01b568f: Status 404 returned error can't find the container with id b29229c861de515f82f2b6a9af7ee906b0efb0900ea7417cbe71925af01b568f
Jul 07 02:14:47.662853 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.662834    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b4d79695c-592zr"]
Jul 07 02:14:47.665486 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.665463    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:14:47.665730 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:47.665710    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1e295e_29cc_4033_be75_609fffee5be9.slice/crio-a3c2597ef9e086dd3c62a7e20f9666534515816fe55230b34dce4476481d3cee WatchSource:0}: Error finding container a3c2597ef9e086dd3c62a7e20f9666534515816fe55230b34dce4476481d3cee: Status 404 returned error can't find the container with id a3c2597ef9e086dd3c62a7e20f9666534515816fe55230b34dce4476481d3cee
Jul 07 02:14:47.831964 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.831922    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:14:47.839877 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.839280    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.839877 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.839404    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:47.842554 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.842529    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 02:14:47.842848 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.842833    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 02:14:47.843234 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.843081    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 02:14:47.843347 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.843267    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 02:14:47.843413 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.843367    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 02:14:47.843687 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.843669    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 02:14:47.844764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.843874    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 02:14:47.844764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.844083    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-brqhh\""
Jul 07 02:14:47.844764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.844283    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 02:14:47.844764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.844480    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 02:14:47.845011 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.844959    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8m6ti1p4u2q3u\""
Jul 07 02:14:47.845061 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.845039    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 02:14:47.845207 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.845176    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 02:14:47.845482 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.845348    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 02:14:47.847272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.847094    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 02:14:47.849982 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.849962    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 02:14:47.852715 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.852635    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995645    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995681    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995855    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995897    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995955    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.995990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.995988    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996011    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfxhb\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-kube-api-access-bfxhb\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996032    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996060    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996582    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996625    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996668    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-config\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:47.997168 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:47.996761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.098429 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098306    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.098429 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098352    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.098429 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098386    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.098429 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098460    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098518    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-config\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098568    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098605    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098629    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098691    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098789    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098816    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099000 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098861    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfxhb\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-kube-api-access-bfxhb\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099780 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.098888    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.099780 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.099092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.100565 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.100162    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.101207 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.100922    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.101812 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.101513    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.101812 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.101636    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.107266 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.101952    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.107266 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.106910    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.108033 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.107941    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.113847 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.113669    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.123370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.123294    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.123370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.123336    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.124846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.124787    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.126751    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.126881    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.126902    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.127085    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.127184    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-config\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.127369    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.127698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.127659    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.128071 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.127791    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfxhb\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-kube-api-access-bfxhb\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.129136 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.129063    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.155219 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.155178    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:14:48.373774 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.373609    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:14:48.385925 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.384887    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9zcvr"
Jul 07 02:14:48.444064 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.444026    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b584fdc-9dfx9" event={"ID":"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60","Type":"ContainerStarted","Data":"1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197"}
Jul 07 02:14:48.444237 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.444088    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b584fdc-9dfx9" event={"ID":"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60","Type":"ContainerStarted","Data":"2e5af1f88b8e2cc50ff07698a060fcab07b81e25397ae19bae02427629cf1e07"}
Jul 07 02:14:48.451793 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.451697    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k" event={"ID":"5aa3034f-8e93-4e74-907c-be6fa9bc1085","Type":"ContainerStarted","Data":"b29229c861de515f82f2b6a9af7ee906b0efb0900ea7417cbe71925af01b568f"}
Jul 07 02:14:48.455053 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.454969    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"c4023628d890b079824216f421c149bb1ff2541e31b3fe74c658c3b0ecd7ae53"}
Jul 07 02:14:48.461614 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.461171    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"a3c2597ef9e086dd3c62a7e20f9666534515816fe55230b34dce4476481d3cee"}
Jul 07 02:14:48.461614 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.461260    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:48.464480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.464141    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689b584fdc-9dfx9" podStartSLOduration=9.464124197 podStartE2EDuration="9.464124197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:48.462069096 +0000 UTC m=+298.665489840" watchObservedRunningTime="2026-07-07 02:14:48.464124197 +0000 UTC m=+298.667544923"
Jul 07 02:14:48.464887 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.464836    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-vk9xl"
Jul 07 02:14:48.483098 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.482850    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-vk9xl" podStartSLOduration=2.219761165 podStartE2EDuration="22.482829084s" totalImagesPullingTime="20.263067919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:48.481298648 +0000 UTC m=+298.684719392" watchObservedRunningTime="2026-07-07 02:14:48.482829084 +0000 UTC m=+298.686249833"
Jul 07 02:14:48.545078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.544790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:48.645808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:48.645690    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:49.024272 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:49.024236    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c22cc3a_4ad5_412e_ba52_209a5905c981.slice/crio-cd1ee8a90bac4e50f8913ba90adc7e9dbb6ec9ba4ddf12b6ee10fbd2ba9d4092 WatchSource:0}: Error finding container cd1ee8a90bac4e50f8913ba90adc7e9dbb6ec9ba4ddf12b6ee10fbd2ba9d4092: Status 404 returned error can't find the container with id cd1ee8a90bac4e50f8913ba90adc7e9dbb6ec9ba4ddf12b6ee10fbd2ba9d4092
Jul 07 02:14:49.471293 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:49.470553    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"cd1ee8a90bac4e50f8913ba90adc7e9dbb6ec9ba4ddf12b6ee10fbd2ba9d4092"}
Jul 07 02:14:49.605563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:49.605192    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:49.605563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:49.605320    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:49.612961 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:49.612933    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:49.853327 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:49.853289    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:49.956998 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:49.956947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:50.054233 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:50.054190    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:50.153635 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:50.153530    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:50.478726 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:50.478543    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:14:51.886787 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:51.522133    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr" podUID="58d76295-5133-4a09-a23b-18ec6b3ed6fb" containerName="registry" containerID="cri-o://2c5a6abc37074ad49f26ad9a826c5b4c64c9adb743202e7c1c895ee3d832f91b" gracePeriod=30
Jul 07 02:14:52.073224 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.073185    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:52.414023 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.413999    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:14:52.417248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.417229    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:14:52.437688 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.437667    2568 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 02:14:52.487787 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.487752    2568 generic.go:356] "Generic (PLEG): container finished" podID="58d76295-5133-4a09-a23b-18ec6b3ed6fb" containerID="2c5a6abc37074ad49f26ad9a826c5b4c64c9adb743202e7c1c895ee3d832f91b" exitCode=0
Jul 07 02:14:52.487893 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.487875    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr" event={"ID":"58d76295-5133-4a09-a23b-18ec6b3ed6fb","Type":"ContainerDied","Data":"2c5a6abc37074ad49f26ad9a826c5b4c64c9adb743202e7c1c895ee3d832f91b"}
Jul 07 02:14:52.553835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.553802    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:52.650667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650429    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650484    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-trusted-ca\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650550    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d76295-5133-4a09-a23b-18ec6b3ed6fb-ca-trust-extracted\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650589    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-image-registry-private-configuration\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650660    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-bound-sa-token\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650691    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfsgh\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-kube-api-access-hfsgh\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650735    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-certificates\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.650943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.650783    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-installation-pull-secrets\") pod \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\" (UID: \"58d76295-5133-4a09-a23b-18ec6b3ed6fb\") "
Jul 07 02:14:52.651176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.651148    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:14:52.651298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.651218    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-trusted-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.652393 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.652146    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:14:52.654621 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.654460    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:14:52.654789 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.654742    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:14:52.655102 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.655071    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:14:52.655556 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.655526    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-kube-api-access-hfsgh" (OuterVolumeSpecName: "kube-api-access-hfsgh") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "kube-api-access-hfsgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:14:52.655765 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.655732    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:14:52.663103 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.663076    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58d76295-5133-4a09-a23b-18ec6b3ed6fb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "58d76295-5133-4a09-a23b-18ec6b3ed6fb" (UID: "58d76295-5133-4a09-a23b-18ec6b3ed6fb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:14:52.752398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752345    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-installation-pull-secrets\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.752398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752376    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.752398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752387    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d76295-5133-4a09-a23b-18ec6b3ed6fb-ca-trust-extracted\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.752398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752396    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d76295-5133-4a09-a23b-18ec6b3ed6fb-image-registry-private-configuration\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.752733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752409    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-bound-sa-token\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.752733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752424    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfsgh\" (UniqueName: \"kubernetes.io/projected/58d76295-5133-4a09-a23b-18ec6b3ed6fb-kube-api-access-hfsgh\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:52.752733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:52.752437    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d76295-5133-4a09-a23b-18ec6b3ed6fb-registry-certificates\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:14:53.031074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.031051    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:14:53.472398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.472360    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:53.494432 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.494392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z" event={"ID":"4dfd0163-83d3-415b-a97b-ea13e64a2afb","Type":"ContainerStarted","Data":"881ebd228898161310cfdcb130868f4557fd5b85171f5201adb3ef7db2592928"}
Jul 07 02:14:53.494600 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.494536    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:53.496350 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.496316    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b" exitCode=0
Jul 07 02:14:53.496461 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.496405    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"}
Jul 07 02:14:53.500272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.499277    2568 generic.go:356] "Generic (PLEG): container finished" podID="e8fbc0dc-24ac-4580-8c92-52773eb35827" containerID="1af5bb78e1e2cfd1b6d2fda300b19d571f72c80a255271a227e1538cac75d9e6" exitCode=0
Jul 07 02:14:53.500272 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.499337    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kw7tf" event={"ID":"e8fbc0dc-24ac-4580-8c92-52773eb35827","Type":"ContainerDied","Data":"1af5bb78e1e2cfd1b6d2fda300b19d571f72c80a255271a227e1538cac75d9e6"}
Jul 07 02:14:53.501089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.501066    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z"
Jul 07 02:14:53.502146 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.502129    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr"
Jul 07 02:14:53.502146 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.502134    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fbfc4f44-6r7cr" event={"ID":"58d76295-5133-4a09-a23b-18ec6b3ed6fb","Type":"ContainerDied","Data":"0a3e0255554b6e57680bd537b9a6f8ac3f92f66764a14498e11773e495296388"}
Jul 07 02:14:53.502292 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.502178    2568 scope.go:117] "RemoveContainer" containerID="2c5a6abc37074ad49f26ad9a826c5b4c64c9adb743202e7c1c895ee3d832f91b"
Jul 07 02:14:53.504904 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.504874    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k" event={"ID":"5aa3034f-8e93-4e74-907c-be6fa9bc1085","Type":"ContainerStarted","Data":"e79882404b0a62214b1ef0acddd9bc699ea7aa856ace0e9a895dc2bfc488d378"}
Jul 07 02:14:53.507752 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.507011    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e" exitCode=0
Jul 07 02:14:53.507752 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.507117    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"}
Jul 07 02:14:53.512005 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.511960    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-89j5z" podStartSLOduration=2.599413694 podStartE2EDuration="7.511948432s" totalImagesPullingTime="4.912534738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:53.511089002 +0000 UTC m=+303.714509749" watchObservedRunningTime="2026-07-07 02:14:53.511948432 +0000 UTC m=+303.715369175"
Jul 07 02:14:53.517264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.517051    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"40d734413ee0e87207cfd8d5db5cff96b872b2d62d9d4f598462569def5b5e63"}
Jul 07 02:14:53.517375 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.517211    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"5f26c0cb14eae74c4647a22ed82a78d59ff93c076050e993161746b65e3b3334"}
Jul 07 02:14:53.517375 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.517363    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"a54131282927243ea4db6cac0c8cc6d65772478367dabcae3baa577319e40468"}
Jul 07 02:14:53.548353 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.548327    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:53.573657 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.573559    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k" podStartSLOduration=3.84681138 podStartE2EDuration="8.573543951s" totalImagesPullingTime="4.726732571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:53.572847255 +0000 UTC m=+303.776268002" watchObservedRunningTime="2026-07-07 02:14:53.573543951 +0000 UTC m=+303.776964673"
Jul 07 02:14:53.634411 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.634381    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75fbfc4f44-6r7cr"]
Jul 07 02:14:53.637552 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:53.637528    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75fbfc4f44-6r7cr"]
Jul 07 02:14:54.433126 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:54.433093    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d76295-5133-4a09-a23b-18ec6b3ed6fb" path="/var/lib/kubelet/pods/58d76295-5133-4a09-a23b-18ec6b3ed6fb/volumes"
Jul 07 02:14:54.536717 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:54.536667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kw7tf" event={"ID":"e8fbc0dc-24ac-4580-8c92-52773eb35827","Type":"ContainerStarted","Data":"e9d4720d445a73fe999edcf09bb4817f7a5cadad3854a05863486e80c4f4780c"}
Jul 07 02:14:54.536921 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:54.536734    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kw7tf" event={"ID":"e8fbc0dc-24ac-4580-8c92-52773eb35827","Type":"ContainerStarted","Data":"c640a1bd2588ac164adba23cdc2bcfed3c40b5b87301b1598e8191625d850b00"}
Jul 07 02:14:54.558131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:54.558071    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kw7tf" podStartSLOduration=8.349760731 podStartE2EDuration="13.558047977s" totalImagesPullingTime="5.093272673s" totalInitContainerRuntime="115.014573ms" isStatefulPod=true podCreationTimestamp="2026-07-07 02:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:54.555076495 +0000 UTC m=+304.758497239" watchObservedRunningTime="2026-07-07 02:14:54.558047977 +0000 UTC m=+304.761468722"
Jul 07 02:14:55.549882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.549839    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"15ec07740ab6b44d57f04d13ceeab97f2399de4cc62e1cc292769366881f3ace"}
Jul 07 02:14:55.550371 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.550000    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"e794888ed985d062162c60cbfb57d4c9c91f06d60d5e44d30b61799c091c9c51"}
Jul 07 02:14:55.550371 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.550051    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" event={"ID":"6c1e295e-29cc-4033-be75-609fffee5be9","Type":"ContainerStarted","Data":"1ee5c3da9b7b85fddb17e5fbc59f7a940e12e9f9574d95f853abfd6334083eed"}
Jul 07 02:14:55.550371 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.550144    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:14:55.574675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.574352    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr" podStartSLOduration=6.4068736 podStartE2EDuration="12.574330192s" totalImagesPullingTime="6.167456592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:55.571473051 +0000 UTC m=+305.774893796" watchObservedRunningTime="2026-07-07 02:14:55.574330192 +0000 UTC m=+305.777750938"
Jul 07 02:14:55.868736 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.868645    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:55.930882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:55.930843    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:56.629885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.629850    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c46cb6c-qps6t"]
Jul 07 02:14:56.630351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.630334    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d76295-5133-4a09-a23b-18ec6b3ed6fb" containerName="registry"
Jul 07 02:14:56.630429 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.630362    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d76295-5133-4a09-a23b-18ec6b3ed6fb" containerName="registry"
Jul 07 02:14:56.630429 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.630426    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d76295-5133-4a09-a23b-18ec6b3ed6fb" containerName="registry"
Jul 07 02:14:56.653593 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.653562    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c46cb6c-qps6t"]
Jul 07 02:14:56.653748 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.653680    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.694995 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.694960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-oauth-config\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.695106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.695020    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-config\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.695106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.695049    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-oauth-serving-cert\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.695106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.695079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-trusted-ca-bundle\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.695241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.695116    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-serving-cert\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.695241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.695197    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-service-ca\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.695334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.695241    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx9vn\" (UniqueName: \"kubernetes.io/projected/93cbaa45-7369-4e5b-9179-4376eeaa473e-kube-api-access-vx9vn\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796278 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796250    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-service-ca\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796423 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796299    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx9vn\" (UniqueName: \"kubernetes.io/projected/93cbaa45-7369-4e5b-9179-4376eeaa473e-kube-api-access-vx9vn\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796479    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-oauth-config\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796597 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796561    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-config\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-oauth-serving-cert\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-trusted-ca-bundle\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.796712 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.796668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-serving-cert\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.797045 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.797020    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-service-ca\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.797203 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.797182    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-config\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.797410 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.797378    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-oauth-serving-cert\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.797468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.797452    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-trusted-ca-bundle\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.798952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.798931    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-oauth-config\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.799380 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.799355    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-serving-cert\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.804450 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.804432    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx9vn\" (UniqueName: \"kubernetes.io/projected/93cbaa45-7369-4e5b-9179-4376eeaa473e-kube-api-access-vx9vn\") pod \"console-77c46cb6c-qps6t\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") " pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:56.977370 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:56.977279    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:14:57.561072 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:57.561041    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"}
Jul 07 02:14:57.561258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:57.561121    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"}
Jul 07 02:14:57.561258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:57.561165    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"}
Jul 07 02:14:58.136060 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.136040    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c46cb6c-qps6t"]
Jul 07 02:14:58.144711 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:14:58.144671    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cbaa45_7369_4e5b_9179_4376eeaa473e.slice/crio-b10af68d2cef9aa2ca0fdb1de202d765e12b681561d91754da11a9b3dcc34006 WatchSource:0}: Error finding container b10af68d2cef9aa2ca0fdb1de202d765e12b681561d91754da11a9b3dcc34006: Status 404 returned error can't find the container with id b10af68d2cef9aa2ca0fdb1de202d765e12b681561d91754da11a9b3dcc34006
Jul 07 02:14:58.568906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.568878    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"}
Jul 07 02:14:58.572593 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.572564    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"}
Jul 07 02:14:58.572707 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.572627    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"}
Jul 07 02:14:58.572707 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.572670    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerStarted","Data":"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"}
Jul 07 02:14:58.574289 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.574264    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c46cb6c-qps6t" event={"ID":"93cbaa45-7369-4e5b-9179-4376eeaa473e","Type":"ContainerStarted","Data":"e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971"}
Jul 07 02:14:58.574391 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.574312    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c46cb6c-qps6t" event={"ID":"93cbaa45-7369-4e5b-9179-4376eeaa473e","Type":"ContainerStarted","Data":"b10af68d2cef9aa2ca0fdb1de202d765e12b681561d91754da11a9b3dcc34006"}
Jul 07 02:14:58.605503 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.605428    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.019372157 podStartE2EDuration="16.605405826s" totalImagesPullingTime="7.566413781s" totalInitContainerRuntime="19.619888ms" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:58.603119892 +0000 UTC m=+308.806540641" watchObservedRunningTime="2026-07-07 02:14:58.605405826 +0000 UTC m=+308.808826571"
Jul 07 02:14:58.624205 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.624144    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77c46cb6c-qps6t" podStartSLOduration=2.624124591 podStartE2EDuration="2.624124591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:58.623295312 +0000 UTC m=+308.826716058" watchObservedRunningTime="2026-07-07 02:14:58.624124591 +0000 UTC m=+308.827545338"
Jul 07 02:14:58.847280 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.847254    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:58.903915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:58.903881    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:14:59.427557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.427525    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:14:59.582074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.582034    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"}
Jul 07 02:14:59.582260 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.582104    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"}
Jul 07 02:14:59.582260 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.582139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"}
Jul 07 02:14:59.582260 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.582165    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"}
Jul 07 02:14:59.582260 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.582190    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerStarted","Data":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"}
Jul 07 02:14:59.613420 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:14:59.613370    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.126744007 podStartE2EDuration="12.613352502s" totalImagesPullingTime="8.458604779s" totalInitContainerRuntime="28.003716ms" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:59.612400899 +0000 UTC m=+309.815821682" watchObservedRunningTime="2026-07-07 02:14:59.613352502 +0000 UTC m=+309.816773249"
Jul 07 02:15:00.562485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:00.562448    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:00.623763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:00.623731    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:01.130863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:01.130824    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:01.231434 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:01.231392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:01.560425 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:01.560397    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b4d79695c-592zr"
Jul 07 02:15:03.155930 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:03.155897    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:15:06.299399 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:06.299364    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:15:06.299887 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:06.299653    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:15:06.977733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:06.977694    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:15:06.977942 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:06.977756    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:15:06.982611 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:06.982571    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:15:07.619299 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:07.619273    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:15:07.666612 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:07.666575    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689b584fdc-9dfx9"]
Jul 07 02:15:07.767728 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:07.767692    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:07.868946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:07.868908    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:07.924993 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:07.924906    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:12.451128 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.451091    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79b66c9fc6-fd545" podUID="881889e3-fc82-4d70-9c62-2619740e2615" containerName="console" containerID="cri-o://c1bc6de6556486d097093b13c06dd3f84fa7e51cd84458ba33ca7442f2ba3685" gracePeriod=15
Jul 07 02:15:12.639733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.639705    2568 generic.go:356] "Generic (PLEG): container finished" podID="881889e3-fc82-4d70-9c62-2619740e2615" containerID="c1bc6de6556486d097093b13c06dd3f84fa7e51cd84458ba33ca7442f2ba3685" exitCode=0
Jul 07 02:15:12.639884 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.639783    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b66c9fc6-fd545" event={"ID":"881889e3-fc82-4d70-9c62-2619740e2615","Type":"ContainerDied","Data":"c1bc6de6556486d097093b13c06dd3f84fa7e51cd84458ba33ca7442f2ba3685"}
Jul 07 02:15:12.729705 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.729676    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:15:12.855242 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855203    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-serving-cert\") pod \"881889e3-fc82-4d70-9c62-2619740e2615\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") "
Jul 07 02:15:12.855443 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855262    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-service-ca\") pod \"881889e3-fc82-4d70-9c62-2619740e2615\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") "
Jul 07 02:15:12.855443 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855303    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-console-config\") pod \"881889e3-fc82-4d70-9c62-2619740e2615\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") "
Jul 07 02:15:12.855443 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855407    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-oauth-serving-cert\") pod \"881889e3-fc82-4d70-9c62-2619740e2615\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") "
Jul 07 02:15:12.855648 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855461    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-oauth-config\") pod \"881889e3-fc82-4d70-9c62-2619740e2615\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") "
Jul 07 02:15:12.855648 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855569    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp78s\" (UniqueName: \"kubernetes.io/projected/881889e3-fc82-4d70-9c62-2619740e2615-kube-api-access-pp78s\") pod \"881889e3-fc82-4d70-9c62-2619740e2615\" (UID: \"881889e3-fc82-4d70-9c62-2619740e2615\") "
Jul 07 02:15:12.855750 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855687    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-console-config" (OuterVolumeSpecName: "console-config") pod "881889e3-fc82-4d70-9c62-2619740e2615" (UID: "881889e3-fc82-4d70-9c62-2619740e2615"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:12.855750 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855713    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-service-ca" (OuterVolumeSpecName: "service-ca") pod "881889e3-fc82-4d70-9c62-2619740e2615" (UID: "881889e3-fc82-4d70-9c62-2619740e2615"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:12.855859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855800    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "881889e3-fc82-4d70-9c62-2619740e2615" (UID: "881889e3-fc82-4d70-9c62-2619740e2615"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:12.855923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855906    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:12.855923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855922    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-console-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:12.856026 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.855930    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/881889e3-fc82-4d70-9c62-2619740e2615-oauth-serving-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:12.857813 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.857788    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "881889e3-fc82-4d70-9c62-2619740e2615" (UID: "881889e3-fc82-4d70-9c62-2619740e2615"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:15:12.857813 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.857796    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881889e3-fc82-4d70-9c62-2619740e2615-kube-api-access-pp78s" (OuterVolumeSpecName: "kube-api-access-pp78s") pod "881889e3-fc82-4d70-9c62-2619740e2615" (UID: "881889e3-fc82-4d70-9c62-2619740e2615"). InnerVolumeSpecName "kube-api-access-pp78s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:15:12.857813 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.857804    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "881889e3-fc82-4d70-9c62-2619740e2615" (UID: "881889e3-fc82-4d70-9c62-2619740e2615"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:15:12.956739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.956648    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-oauth-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:12.956739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.956680    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp78s\" (UniqueName: \"kubernetes.io/projected/881889e3-fc82-4d70-9c62-2619740e2615-kube-api-access-pp78s\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:12.956739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:12.956692    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/881889e3-fc82-4d70-9c62-2619740e2615-console-serving-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:13.645221 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:13.645191    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b66c9fc6-fd545"
Jul 07 02:15:13.645674 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:13.645222    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b66c9fc6-fd545" event={"ID":"881889e3-fc82-4d70-9c62-2619740e2615","Type":"ContainerDied","Data":"c7b6b45aac790c2d1106816eb2238e0ae666b0a6ca5a236377090e6f6fcd3918"}
Jul 07 02:15:13.645674 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:13.645277    2568 scope.go:117] "RemoveContainer" containerID="c1bc6de6556486d097093b13c06dd3f84fa7e51cd84458ba33ca7442f2ba3685"
Jul 07 02:15:13.665883 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:13.665847    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79b66c9fc6-fd545"]
Jul 07 02:15:13.669875 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:13.669851    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79b66c9fc6-fd545"]
Jul 07 02:15:14.432948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:14.432912    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881889e3-fc82-4d70-9c62-2619740e2615" path="/var/lib/kubelet/pods/881889e3-fc82-4d70-9c62-2619740e2615/volumes"
Jul 07 02:15:15.598160 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:15.598119    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:15.674869 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:15.674838    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:17.669476 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:17.669439    2568 generic.go:356] "Generic (PLEG): container finished" podID="3ff63f40-bce3-427c-990a-9545b6cc9d5b" containerID="08e29ea16adf599baf32fe60767bf002d69452094cb11d17ef553382305fcb90" exitCode=0
Jul 07 02:15:17.669940 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:17.669534    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr" event={"ID":"3ff63f40-bce3-427c-990a-9545b6cc9d5b","Type":"ContainerDied","Data":"08e29ea16adf599baf32fe60767bf002d69452094cb11d17ef553382305fcb90"}
Jul 07 02:15:17.670003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:17.669968    2568 scope.go:117] "RemoveContainer" containerID="08e29ea16adf599baf32fe60767bf002d69452094cb11d17ef553382305fcb90"
Jul 07 02:15:18.676241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:18.676208    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4tnxr" event={"ID":"3ff63f40-bce3-427c-990a-9545b6cc9d5b","Type":"ContainerStarted","Data":"6a43f6e618b8e29b730385d4893950f59e654eed5b67c37785f592566fa21c15"}
Jul 07 02:15:26.304735 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:26.304708    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:15:26.308507 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:26.308472    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5cbfc84df-p225k"
Jul 07 02:15:27.714765 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:27.714734    2568 generic.go:356] "Generic (PLEG): container finished" podID="a77aff70-34c1-4ab4-a4be-901ecebc8a7c" containerID="cb1fc5cc601540b3bcbe2abbb6803c1f0ed80ed2ee558823aa1d246e211acb85" exitCode=0
Jul 07 02:15:27.715113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:27.714804    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb" event={"ID":"a77aff70-34c1-4ab4-a4be-901ecebc8a7c","Type":"ContainerDied","Data":"cb1fc5cc601540b3bcbe2abbb6803c1f0ed80ed2ee558823aa1d246e211acb85"}
Jul 07 02:15:27.715167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:27.715154    2568 scope.go:117] "RemoveContainer" containerID="cb1fc5cc601540b3bcbe2abbb6803c1f0ed80ed2ee558823aa1d246e211acb85"
Jul 07 02:15:28.710952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:28.710909    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xt4rs_a387479e-f772-49bc-b157-c146c9358d3e/serve-healthcheck-canary/0.log"
Jul 07 02:15:28.721134 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:28.721095    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7prkb" event={"ID":"a77aff70-34c1-4ab4-a4be-901ecebc8a7c","Type":"ContainerStarted","Data":"8df5e41fbac96732e53f1be126619d9810d49f6fb3555b0e45cce8c744facd1d"}
Jul 07 02:15:30.564645 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:30.564609    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:30.641194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:30.641160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:32.688923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:32.688877    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-689b584fdc-9dfx9" podUID="a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" containerName="console" containerID="cri-o://1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197" gracePeriod=15
Jul 07 02:15:32.983074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:32.983044    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:15:33.144718 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144684    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-service-ca\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.144906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144749    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-serving-cert\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.144906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144789    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-trusted-ca-bundle\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.144906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144817    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbbk5\" (UniqueName: \"kubernetes.io/projected/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-kube-api-access-zbbk5\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.144906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144842    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-oauth-serving-cert\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.145121 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144917    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-oauth-config\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.145121 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.144954    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-config\") pod \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\" (UID: \"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60\") "
Jul 07 02:15:33.145227 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.145113    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-service-ca" (OuterVolumeSpecName: "service-ca") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:33.145346 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.145302    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.145478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.145390    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:33.145478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.145403    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:33.145639 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.145533    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-config" (OuterVolumeSpecName: "console-config") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:15:33.147175 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.147151    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:15:33.147265 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.147213    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-kube-api-access-zbbk5" (OuterVolumeSpecName: "kube-api-access-zbbk5") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "kube-api-access-zbbk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:15:33.147265 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.147213    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" (UID: "a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:15:33.246244 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.246220    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-oauth-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.246382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.246246    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.246382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.246269    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-console-serving-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.246382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.246278    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-trusted-ca-bundle\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.246382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.246287    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbbk5\" (UniqueName: \"kubernetes.io/projected/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-kube-api-access-zbbk5\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.246382 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.246296    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60-oauth-serving-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:15:33.743592 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.743556    2568 generic.go:356] "Generic (PLEG): container finished" podID="a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" containerID="1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197" exitCode=0
Jul 07 02:15:33.744056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.743635    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b584fdc-9dfx9"
Jul 07 02:15:33.744056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.743646    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b584fdc-9dfx9" event={"ID":"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60","Type":"ContainerDied","Data":"1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197"}
Jul 07 02:15:33.744056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.743753    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b584fdc-9dfx9" event={"ID":"a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60","Type":"ContainerDied","Data":"2e5af1f88b8e2cc50ff07698a060fcab07b81e25397ae19bae02427629cf1e07"}
Jul 07 02:15:33.744056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.743784    2568 scope.go:117] "RemoveContainer" containerID="1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197"
Jul 07 02:15:33.753368 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.753345    2568 scope.go:117] "RemoveContainer" containerID="1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197"
Jul 07 02:15:33.753649 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:15:33.753630    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197\": container with ID starting with 1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197 not found: ID does not exist" containerID="1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197"
Jul 07 02:15:33.753714 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.753659    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197"} err="failed to get container status \"1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197\": rpc error: code = NotFound desc = could not find container \"1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197\": container with ID starting with 1d8eb26fa1489156b626b9eada6e4c0d8a74021bcfd668e752c91806f5c52197 not found: ID does not exist"
Jul 07 02:15:33.764563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.764541    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689b584fdc-9dfx9"]
Jul 07 02:15:33.768295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:33.768271    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689b584fdc-9dfx9"]
Jul 07 02:15:34.432763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:34.432732    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" path="/var/lib/kubelet/pods/a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60/volumes"
Jul 07 02:15:38.168202 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:38.168165    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:38.269105 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:38.269071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:38.370180 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:38.370139    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:47.070061 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:47.069974    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:47.132186 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:47.132157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:48.155388 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:48.155348    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:15:48.176062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:48.176037    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:15:48.824184 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:48.824159    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:15:57.720957 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:57.720923    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:57.779351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:57.779318    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:59.212052 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:59.212017    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:15:59.288638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:15:59.288605    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:00.907196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:00.907163    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:00.969781 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:00.969752    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:02.024242 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024207    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:16:02.024771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024464    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="alertmanager" containerID="cri-o://0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9" gracePeriod=120
Jul 07 02:16:02.024771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024547    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="prom-label-proxy" containerID="cri-o://61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71" gracePeriod=120
Jul 07 02:16:02.024771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024587    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-metric" containerID="cri-o://12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81" gracePeriod=120
Jul 07 02:16:02.024771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024588    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy" containerID="cri-o://bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6" gracePeriod=120
Jul 07 02:16:02.024771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024545    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-web" containerID="cri-o://3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359" gracePeriod=120
Jul 07 02:16:02.024771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.024601    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="config-reloader" containerID="cri-o://11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39" gracePeriod=120
Jul 07 02:16:02.873981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.873949    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71" exitCode=0
Jul 07 02:16:02.873981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.873975    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6" exitCode=0
Jul 07 02:16:02.873981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.873981    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39" exitCode=0
Jul 07 02:16:02.873981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.873988    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9" exitCode=0
Jul 07 02:16:02.874241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.874016    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"}
Jul 07 02:16:02.874241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.874086    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"}
Jul 07 02:16:02.874241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.874117    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"}
Jul 07 02:16:02.874241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:02.874142    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"}
Jul 07 02:16:03.279245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.279221    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:03.409248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409172    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-out\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409214    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409238    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409295    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44hgt\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-kube-api-access-44hgt\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409311    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-volume\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409330    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-cluster-tls-config\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409385    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409762 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409739    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-metrics-client-ca\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409823 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409798    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-tls-assets\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409872 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409826    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-main-db\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409872 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409857    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409970 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409890    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-web-config\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.409970 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.409926    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-trusted-ca-bundle\") pod \"7d23ab93-ef4c-4961-a780-c036ed541bbe\" (UID: \"7d23ab93-ef4c-4961-a780-c036ed541bbe\") "
Jul 07 02:16:03.410932 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.410651    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:03.411435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.411193    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:03.412123 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412093    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.412226 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412135    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-volume" (OuterVolumeSpecName: "config-volume") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.412226 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412158    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.412374 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412349    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-kube-api-access-44hgt" (OuterVolumeSpecName: "kube-api-access-44hgt") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "kube-api-access-44hgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:16:03.412441 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412417    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:16:03.412524 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412477    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.412620 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.412600    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-out" (OuterVolumeSpecName: "config-out") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:16:03.413587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.413555    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:16:03.413884 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.413862    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.416891 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.416782    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.423892 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.423868    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-web-config" (OuterVolumeSpecName: "web-config") pod "7d23ab93-ef4c-4961-a780-c036ed541bbe" (UID: "7d23ab93-ef4c-4961-a780-c036ed541bbe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:03.510951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510925    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44hgt\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-kube-api-access-44hgt\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.510951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510949    2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-volume\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510958    2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-cluster-tls-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510968    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510976    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-metrics-client-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510985    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d23ab93-ef4c-4961-a780-c036ed541bbe-tls-assets\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.510993    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-main-db\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.511004    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-main-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.511012    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-web-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.511021    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d23ab93-ef4c-4961-a780-c036ed541bbe-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.511029    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d23ab93-ef4c-4961-a780-c036ed541bbe-config-out\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.511037    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.511113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.511046    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d23ab93-ef4c-4961-a780-c036ed541bbe-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:03.880724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880688    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81" exitCode=0
Jul 07 02:16:03.880724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880717    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerID="3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359" exitCode=0
Jul 07 02:16:03.880934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880753    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:03.880934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880774    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"}
Jul 07 02:16:03.880934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880845    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"}
Jul 07 02:16:03.880934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d23ab93-ef4c-4961-a780-c036ed541bbe","Type":"ContainerDied","Data":"c4023628d890b079824216f421c149bb1ff2541e31b3fe74c658c3b0ecd7ae53"}
Jul 07 02:16:03.880934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.880903    2568 scope.go:117] "RemoveContainer" containerID="61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"
Jul 07 02:16:03.888940 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.888923    2568 scope.go:117] "RemoveContainer" containerID="12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"
Jul 07 02:16:03.895650 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.895635    2568 scope.go:117] "RemoveContainer" containerID="bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"
Jul 07 02:16:03.902014 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.901985    2568 scope.go:117] "RemoveContainer" containerID="3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"
Jul 07 02:16:03.904801 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.904778    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:16:03.911186 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.911160    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:16:03.916295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.916277    2568 scope.go:117] "RemoveContainer" containerID="11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"
Jul 07 02:16:03.923241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.923227    2568 scope.go:117] "RemoveContainer" containerID="0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"
Jul 07 02:16:03.929862 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.929846    2568 scope.go:117] "RemoveContainer" containerID="6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"
Jul 07 02:16:03.936834 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.936809    2568 scope.go:117] "RemoveContainer" containerID="61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"
Jul 07 02:16:03.937079 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.937060    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71\": container with ID starting with 61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71 not found: ID does not exist" containerID="61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"
Jul 07 02:16:03.937150 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937087    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"} err="failed to get container status \"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71\": rpc error: code = NotFound desc = could not find container \"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71\": container with ID starting with 61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71 not found: ID does not exist"
Jul 07 02:16:03.937150 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937106    2568 scope.go:117] "RemoveContainer" containerID="12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"
Jul 07 02:16:03.937333 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.937313    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81\": container with ID starting with 12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81 not found: ID does not exist" containerID="12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"
Jul 07 02:16:03.937419 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937333    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"} err="failed to get container status \"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81\": rpc error: code = NotFound desc = could not find container \"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81\": container with ID starting with 12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81 not found: ID does not exist"
Jul 07 02:16:03.937419 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937346    2568 scope.go:117] "RemoveContainer" containerID="bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"
Jul 07 02:16:03.937419 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937368    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:16:03.937590 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.937541    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6\": container with ID starting with bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6 not found: ID does not exist" containerID="bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"
Jul 07 02:16:03.937590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937561    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"} err="failed to get container status \"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6\": rpc error: code = NotFound desc = could not find container \"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6\": container with ID starting with bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6 not found: ID does not exist"
Jul 07 02:16:03.937590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937575    2568 scope.go:117] "RemoveContainer" containerID="3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"
Jul 07 02:16:03.937767 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.937752    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359\": container with ID starting with 3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359 not found: ID does not exist" containerID="3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"
Jul 07 02:16:03.937804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937768    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"} err="failed to get container status \"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359\": rpc error: code = NotFound desc = could not find container \"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359\": container with ID starting with 3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359 not found: ID does not exist"
Jul 07 02:16:03.937804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937784    2568 scope.go:117] "RemoveContainer" containerID="11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"
Jul 07 02:16:03.937881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937829    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="alertmanager"
Jul 07 02:16:03.937881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937847    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="alertmanager"
Jul 07 02:16:03.937881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937864    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-web"
Jul 07 02:16:03.937881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937873    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-web"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937882    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937891    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937903    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" containerName="console"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937911    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" containerName="console"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937921    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="init-config-reloader"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937930    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="init-config-reloader"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937942    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881889e3-fc82-4d70-9c62-2619740e2615" containerName="console"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937951    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="881889e3-fc82-4d70-9c62-2619740e2615" containerName="console"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937960    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="prom-label-proxy"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937969    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="prom-label-proxy"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937980    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-metric"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.937988    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-metric"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938015    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="config-reloader"
Jul 07 02:16:03.938020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938024    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="config-reloader"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.937984    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39\": container with ID starting with 11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39 not found: ID does not exist" containerID="11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938078    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"} err="failed to get container status \"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39\": rpc error: code = NotFound desc = could not find container \"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39\": container with ID starting with 11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39 not found: ID does not exist"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938093    2568 scope.go:117] "RemoveContainer" containerID="0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938121    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-web"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938132    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="881889e3-fc82-4d70-9c62-2619740e2615" containerName="console"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938143    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="prom-label-proxy"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938152    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy-metric"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938161    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="config-reloader"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938170    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="alertmanager"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938179    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" containerName="kube-rbac-proxy"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938188    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81a3ac3-0df3-404b-ad9f-d4f4c49b2c60" containerName="console"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.938324    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9\": container with ID starting with 0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9 not found: ID does not exist" containerID="0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938350    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"} err="failed to get container status \"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9\": rpc error: code = NotFound desc = could not find container \"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9\": container with ID starting with 0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9 not found: ID does not exist"
Jul 07 02:16:03.938557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938370    2568 scope.go:117] "RemoveContainer" containerID="6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"
Jul 07 02:16:03.939038 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:03.938598    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e\": container with ID starting with 6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e not found: ID does not exist" containerID="6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"
Jul 07 02:16:03.939038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938615    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"} err="failed to get container status \"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e\": rpc error: code = NotFound desc = could not find container \"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e\": container with ID starting with 6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e not found: ID does not exist"
Jul 07 02:16:03.939038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938627    2568 scope.go:117] "RemoveContainer" containerID="61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"
Jul 07 02:16:03.939038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938836    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71"} err="failed to get container status \"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71\": rpc error: code = NotFound desc = could not find container \"61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71\": container with ID starting with 61312dd2425f69f4d67df7cc2ac8e336b34553777510378ec70576ff81909a71 not found: ID does not exist"
Jul 07 02:16:03.939038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.938849    2568 scope.go:117] "RemoveContainer" containerID="12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"
Jul 07 02:16:03.939038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939030    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81"} err="failed to get container status \"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81\": rpc error: code = NotFound desc = could not find container \"12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81\": container with ID starting with 12560e226e26995b1f4516934e73a76152516578613f0b8bac0cf59a85035d81 not found: ID does not exist"
Jul 07 02:16:03.939271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939043    2568 scope.go:117] "RemoveContainer" containerID="bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"
Jul 07 02:16:03.939271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939234    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6"} err="failed to get container status \"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6\": rpc error: code = NotFound desc = could not find container \"bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6\": container with ID starting with bee62569b2357dc6c15491c252bfbdcd4b31edd1c2e20486aa74d14407b476e6 not found: ID does not exist"
Jul 07 02:16:03.939271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939251    2568 scope.go:117] "RemoveContainer" containerID="3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"
Jul 07 02:16:03.939475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939459    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359"} err="failed to get container status \"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359\": rpc error: code = NotFound desc = could not find container \"3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359\": container with ID starting with 3ae91034208dc4b2c90bdc7ec1e19dba9b88c13be3791ee913f71e8331118359 not found: ID does not exist"
Jul 07 02:16:03.939545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939475    2568 scope.go:117] "RemoveContainer" containerID="11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"
Jul 07 02:16:03.939725 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939708    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39"} err="failed to get container status \"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39\": rpc error: code = NotFound desc = could not find container \"11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39\": container with ID starting with 11777d8735bd4bbb0f761a61cb368cebd90f8a97d1f8cf41d66c9b59350b1b39 not found: ID does not exist"
Jul 07 02:16:03.939782 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939726    2568 scope.go:117] "RemoveContainer" containerID="0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"
Jul 07 02:16:03.939923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939904    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9"} err="failed to get container status \"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9\": rpc error: code = NotFound desc = could not find container \"0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9\": container with ID starting with 0da266055c9f5ec56feac38671b88095ca301d5bac4a9c1206cc5b30c3d36dd9 not found: ID does not exist"
Jul 07 02:16:03.939973 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.939924    2568 scope.go:117] "RemoveContainer" containerID="6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"
Jul 07 02:16:03.940115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.940094    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e"} err="failed to get container status \"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e\": rpc error: code = NotFound desc = could not find container \"6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e\": container with ID starting with 6bbfcee530b371305db8559103c942ec584665492f55295a3cfaca7392204f9e not found: ID does not exist"
Jul 07 02:16:03.942180 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.942161    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:03.944782 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.944765    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 02:16:03.944855 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.944767    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 02:16:03.945082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945064    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r9qdg\""
Jul 07 02:16:03.945166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945101    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 02:16:03.945166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945102    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 02:16:03.945166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945154    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 02:16:03.945291 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945269    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 02:16:03.945350 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945311    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 02:16:03.945403 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.945384    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 02:16:03.951090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.951074    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 02:16:03.955822 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:03.955804    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:16:04.015775 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015742    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.015929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.015929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015840    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.015929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015878    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.015929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015899    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-config-out\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.015929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015916    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016105 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.015931    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdg2\" (UniqueName: \"kubernetes.io/projected/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-kube-api-access-7hdg2\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016105 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.016020    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016105 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.016048    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-web-config\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016105 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.016072    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016228 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.016108    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016228 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.016137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.016228 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.016155    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117043 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117049    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117068    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117119    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-config-out\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117164    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdg2\" (UniqueName: \"kubernetes.io/projected/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-kube-api-access-7hdg2\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117213    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117235    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-web-config\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117318    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117341    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117862 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117727    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.117923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.117884    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120464 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120204    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120464 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120248    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120464 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120400    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120464 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120451    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120757 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120520    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120820 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120750    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.120820 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.120790    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-config-out\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.121073 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.121047    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.121619 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.121600    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.122535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.122514    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-web-config\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.125332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.125303    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdg2\" (UniqueName: \"kubernetes.io/projected/ab5ff5b1-2973-40bd-bf04-c82cb710ae9f-kube-api-access-7hdg2\") pod \"alertmanager-main-0\" (UID: \"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.218403 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.218328    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:16:04.220452 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.220425    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/478e76b1-c8b0-466d-aa7e-5dc477d27b41-metrics-certs\") pod \"network-metrics-daemon-mzcc5\" (UID: \"478e76b1-c8b0-466d-aa7e-5dc477d27b41\") " pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:16:04.231164 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.231143    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q64zw\""
Jul 07 02:16:04.239094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.239081    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzcc5"
Jul 07 02:16:04.250926 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.250901    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 02:16:04.386609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.386579    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mzcc5"]
Jul 07 02:16:04.389244 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:16:04.389216    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478e76b1_c8b0_466d_aa7e_5dc477d27b41.slice/crio-333b04d1ce9f4d873e6a882922ae958a5aacffad169a56d3c0ad95056a990363 WatchSource:0}: Error finding container 333b04d1ce9f4d873e6a882922ae958a5aacffad169a56d3c0ad95056a990363: Status 404 returned error can't find the container with id 333b04d1ce9f4d873e6a882922ae958a5aacffad169a56d3c0ad95056a990363
Jul 07 02:16:04.421704 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.421678    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 02:16:04.423365 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:16:04.423338    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5ff5b1_2973_40bd_bf04_c82cb710ae9f.slice/crio-238b45c2e78b2497bbd5f040a1ab98332045295160e7c81242da66c0ef3ea3fe WatchSource:0}: Error finding container 238b45c2e78b2497bbd5f040a1ab98332045295160e7c81242da66c0ef3ea3fe: Status 404 returned error can't find the container with id 238b45c2e78b2497bbd5f040a1ab98332045295160e7c81242da66c0ef3ea3fe
Jul 07 02:16:04.431948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.431921    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d23ab93-ef4c-4961-a780-c036ed541bbe" path="/var/lib/kubelet/pods/7d23ab93-ef4c-4961-a780-c036ed541bbe/volumes"
Jul 07 02:16:04.887117 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.887079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mzcc5" event={"ID":"478e76b1-c8b0-466d-aa7e-5dc477d27b41","Type":"ContainerStarted","Data":"333b04d1ce9f4d873e6a882922ae958a5aacffad169a56d3c0ad95056a990363"}
Jul 07 02:16:04.889952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.889924    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab5ff5b1-2973-40bd-bf04-c82cb710ae9f" containerID="9b9d4a63c53a3214617cc1739e2c20de2b9f37e4cb488acafdeb0dac9c29f1fd" exitCode=0
Jul 07 02:16:04.890103 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.889968    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerDied","Data":"9b9d4a63c53a3214617cc1739e2c20de2b9f37e4cb488acafdeb0dac9c29f1fd"}
Jul 07 02:16:04.890103 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:04.890033    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"238b45c2e78b2497bbd5f040a1ab98332045295160e7c81242da66c0ef3ea3fe"}
Jul 07 02:16:05.905142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.905105    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"30d1d107ff2cc01c7b07cba3c1b328f68d50a6706e9dbd8dff4bbb7395e31a34"}
Jul 07 02:16:05.905664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.905220    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"5ae0f1034d12be4f68b42c16d687d944ae53fbd796c54ff47bc4710f5aacc2af"}
Jul 07 02:16:05.905664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.905269    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"f9dd8ab5ad6a5a744aa006e34d47edffdb75605c33d2b68a1d6668a3d725f3bd"}
Jul 07 02:16:05.905664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.905326    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"04ca685541e739e14c2305c261d849607b3df9e61cc466357b4d1704486c519c"}
Jul 07 02:16:05.905664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.905362    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"4681a41e52410a76acb965a355d2ad6267beb3d39abe3904865c4ad731af60e1"}
Jul 07 02:16:05.905664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.905401    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ab5ff5b1-2973-40bd-bf04-c82cb710ae9f","Type":"ContainerStarted","Data":"e911505c042d14166af80b9375919363a658785616e66755d9a7c59c01c7c10e"}
Jul 07 02:16:05.907227 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.907198    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mzcc5" event={"ID":"478e76b1-c8b0-466d-aa7e-5dc477d27b41","Type":"ContainerStarted","Data":"20c41c7b53f267942750abc16544ffde9850ec939a6fbdd4ee2a6d00a53dd93b"}
Jul 07 02:16:05.907343 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.907247    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mzcc5" event={"ID":"478e76b1-c8b0-466d-aa7e-5dc477d27b41","Type":"ContainerStarted","Data":"902fc684b66cfec34dc4b78ba262b45527f3468a4b1dc01538304545172dd7ef"}
Jul 07 02:16:05.933001 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:05.932945    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.913364689 podStartE2EDuration="2.932924244s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.559555ms" isStatefulPod=false podCreationTimestamp="2026-07-07 02:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:16:05.930278953 +0000 UTC m=+376.133699700" watchObservedRunningTime="2026-07-07 02:16:05.932924244 +0000 UTC m=+376.136344991"
Jul 07 02:16:06.080426 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.080385    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:06.180433 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.180344    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:06.272763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.272708    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mzcc5" podStartSLOduration=375.357845907 podStartE2EDuration="6m16.272688409s" totalImagesPullingTime="914.842502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:16:05.94730764 +0000 UTC m=+376.150728385" watchObservedRunningTime="2026-07-07 02:16:06.272688409 +0000 UTC m=+376.476109198"
Jul 07 02:16:06.272968 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.272867    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:16:06.273154 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.273123    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="prometheus" containerID="cri-o://38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e" gracePeriod=600
Jul 07 02:16:06.273311 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.273150    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy" containerID="cri-o://f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc" gracePeriod=600
Jul 07 02:16:06.273311 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.273178    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-web" containerID="cri-o://b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94" gracePeriod=600
Jul 07 02:16:06.273311 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.273150    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="thanos-sidecar" containerID="cri-o://2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309" gracePeriod=600
Jul 07 02:16:06.273311 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.273208    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="config-reloader" containerID="cri-o://29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1" gracePeriod=600
Jul 07 02:16:06.273311 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.273206    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678" gracePeriod=600
Jul 07 02:16:06.509734 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.509707    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:06.637338 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637304    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-0\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637338 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637344    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-tls-assets\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637379    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-metrics-client-ca\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637407    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-config-out\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637457    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-config\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637563 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637533    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-serving-certs-ca-bundle\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637567    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-1\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637606    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfxhb\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-kube-api-access-bfxhb\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637640    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-grpc-tls\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637673    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-thanos-prometheus-http-client-file\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637713    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-db\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637739    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-trusted-ca-bundle\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637764    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-metrics-client-certs\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.637817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637791    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-kubelet-serving-ca-bundle\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637835    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-2\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637874    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637906    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-kube-rbac-proxy\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637915    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637936    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637945    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-tls\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.638215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.637969    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-web-config\") pod \"2c22cc3a-4ad5-412e-ba52-209a5905c981\" (UID: \"2c22cc3a-4ad5-412e-ba52-209a5905c981\") "
Jul 07 02:16:06.638587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.638250    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-metrics-client-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.638587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.638271    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.638587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.638367    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.638587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.638487    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.638846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.638768    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.639189 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.639022    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.639762 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.639474    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:06.640471 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.640442    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.641096 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.641070    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:16:06.641598 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.641561    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-config-out" (OuterVolumeSpecName: "config-out") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:16:06.641692 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.641654    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-config" (OuterVolumeSpecName: "config") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.641692 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.641677    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.642089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.642062    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.642258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.642223    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:16:06.642258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.642223    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.642421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.642367    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.642951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.642935    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.643019 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.642998    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-kube-api-access-bfxhb" (OuterVolumeSpecName: "kube-api-access-bfxhb") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "kube-api-access-bfxhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:16:06.643571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.643556    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.651663 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.651643    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-web-config" (OuterVolumeSpecName: "web-config") pod "2c22cc3a-4ad5-412e-ba52-209a5905c981" (UID: "2c22cc3a-4ad5-412e-ba52-209a5905c981"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739247    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739281    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-config-out\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739290    2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739298    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739308    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfxhb\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-kube-api-access-bfxhb\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739319    2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-grpc-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739327    2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739336    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-db\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739345    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739353    2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-metrics-client-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739361    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739370    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739378    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739386    2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-kube-rbac-proxy\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739394    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739402    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c22cc3a-4ad5-412e-ba52-209a5905c981-web-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739409    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c22cc3a-4ad5-412e-ba52-209a5905c981-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.739706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.739425    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c22cc3a-4ad5-412e-ba52-209a5905c981-tls-assets\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:06.914052 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914018    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678" exitCode=0
Jul 07 02:16:06.914052 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914047    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc" exitCode=0
Jul 07 02:16:06.914052 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914058    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94" exitCode=0
Jul 07 02:16:06.914052 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914067    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309" exitCode=0
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914074    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1" exitCode=0
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914082    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e" exitCode=0
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914095    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"}
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914120    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914169    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"}
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914212    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"}
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914238    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"}
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914252    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914269    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"}
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914297    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"}
Jul 07 02:16:06.914602 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.914327    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c22cc3a-4ad5-412e-ba52-209a5905c981","Type":"ContainerDied","Data":"cd1ee8a90bac4e50f8913ba90adc7e9dbb6ec9ba4ddf12b6ee10fbd2ba9d4092"}
Jul 07 02:16:06.922535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.922516    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.929608 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.929564    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.936504 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.936478    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.941965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.941946    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:16:06.943896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.943881    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.946887 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.946869    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:16:06.951373 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.951359    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.959015 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.958997    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.965903 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.965884    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.966154 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.966136    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.966206 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966163    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"} err="failed to get container status \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist"
Jul 07 02:16:06.966206 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966182    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.966411 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.966395    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.966455 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966418    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"} err="failed to get container status \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist"
Jul 07 02:16:06.966455 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966436    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.966706 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.966689    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.966749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966712    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"} err="failed to get container status \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist"
Jul 07 02:16:06.966749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966727    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.966935 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.966920    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.966975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966939    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"} err="failed to get container status \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist"
Jul 07 02:16:06.966975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.966953    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.967144 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.967128    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.967179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967147    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"} err="failed to get container status \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist"
Jul 07 02:16:06.967179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967162    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.967347 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.967331    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.967381 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967354    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"} err="failed to get container status \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist"
Jul 07 02:16:06.967381 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967366    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.967596 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:06.967581    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.967641 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967599    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"} err="failed to get container status \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist"
Jul 07 02:16:06.967641 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967613    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.967828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967810    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"} err="failed to get container status \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist"
Jul 07 02:16:06.967828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.967828    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.968045 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968030    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"} err="failed to get container status \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist"
Jul 07 02:16:06.968087 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968045    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.968237 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968218    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"} err="failed to get container status \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist"
Jul 07 02:16:06.968285 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968237    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.968453 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968435    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"} err="failed to get container status \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist"
Jul 07 02:16:06.968518 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968454    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.968718 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968701    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"} err="failed to get container status \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist"
Jul 07 02:16:06.968782 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968719    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.968950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968932    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"} err="failed to get container status \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist"
Jul 07 02:16:06.968993 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.968950    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.969191 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969170    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"} err="failed to get container status \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist"
Jul 07 02:16:06.969244 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969193    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.969422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969405    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"} err="failed to get container status \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist"
Jul 07 02:16:06.969474 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969422    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.969664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969645    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"} err="failed to get container status \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist"
Jul 07 02:16:06.969729 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969667    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.969866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969849    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"} err="failed to get container status \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist"
Jul 07 02:16:06.969918 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.969866    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.970025 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970009    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"} err="failed to get container status \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist"
Jul 07 02:16:06.970067 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970026    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.970212 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970194    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"} err="failed to get container status \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist"
Jul 07 02:16:06.970253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970214    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.970446 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970428    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"} err="failed to get container status \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist"
Jul 07 02:16:06.970484 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970447    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.970816 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970792    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"} err="failed to get container status \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist"
Jul 07 02:16:06.970907 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.970830    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.971167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971140    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"} err="failed to get container status \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist"
Jul 07 02:16:06.971273 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971167    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.971474 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971446    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"} err="failed to get container status \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist"
Jul 07 02:16:06.971474 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971465    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.971742 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971724    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"} err="failed to get container status \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist"
Jul 07 02:16:06.971815 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971754    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.971997 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971980    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"} err="failed to get container status \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist"
Jul 07 02:16:06.972049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.971998    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.972170 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972154    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"} err="failed to get container status \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist"
Jul 07 02:16:06.972214 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972171    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.972365 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972340    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"} err="failed to get container status \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist"
Jul 07 02:16:06.972403 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972368    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.972639 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972623    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"} err="failed to get container status \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist"
Jul 07 02:16:06.972680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972640    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.972900 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972880    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"} err="failed to get container status \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist"
Jul 07 02:16:06.972900 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972900    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.973043 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.972958    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:16:06.973101 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973079    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"} err="failed to get container status \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist"
Jul 07 02:16:06.973144 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973103    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.973313 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973297    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"} err="failed to get container status \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist"
Jul 07 02:16:06.973374 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973313    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.973421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973375    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-thanos"
Jul 07 02:16:06.973421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973389    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-thanos"
Jul 07 02:16:06.973421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973397    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="config-reloader"
Jul 07 02:16:06.973421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973404    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="config-reloader"
Jul 07 02:16:06.973421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973420    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="init-config-reloader"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973426    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="init-config-reloader"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973434    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973439    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973446    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-web"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973452    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-web"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973459    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="thanos-sidecar"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973464    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="thanos-sidecar"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973471    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="prometheus"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973476    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="prometheus"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973556    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-web"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973561    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"} err="failed to get container status \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973586    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973572    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="thanos-sidecar"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973634    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="prometheus"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973641    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="config-reloader"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973650    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy-thanos"
Jul 07 02:16:06.973680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973659    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" containerName="kube-rbac-proxy"
Jul 07 02:16:06.974274 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973834    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"} err="failed to get container status \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist"
Jul 07 02:16:06.974274 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.973847    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.974274 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974050    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"} err="failed to get container status \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist"
Jul 07 02:16:06.974274 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974064    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.974412 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974272    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"} err="failed to get container status \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist"
Jul 07 02:16:06.974412 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974286    2568 scope.go:117] "RemoveContainer" containerID="3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"
Jul 07 02:16:06.974542 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974482    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678"} err="failed to get container status \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": rpc error: code = NotFound desc = could not find container \"3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678\": container with ID starting with 3efba7c970eb15bab24cebde8c1ba69c8272bb0acfaade8dc238d6bbee0a8678 not found: ID does not exist"
Jul 07 02:16:06.974542 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974513    2568 scope.go:117] "RemoveContainer" containerID="f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"
Jul 07 02:16:06.974764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974746    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc"} err="failed to get container status \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": rpc error: code = NotFound desc = could not find container \"f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc\": container with ID starting with f1883af498f17f515700bcc01e44702974161f7153343c6e801917af3bb273fc not found: ID does not exist"
Jul 07 02:16:06.974818 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974764    2568 scope.go:117] "RemoveContainer" containerID="b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"
Jul 07 02:16:06.974952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974936    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94"} err="failed to get container status \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": rpc error: code = NotFound desc = could not find container \"b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94\": container with ID starting with b9ffbeea1ff8f3ff0682e1776bffa683989b9ea8eac68570bad53d3fc3c3be94 not found: ID does not exist"
Jul 07 02:16:06.974989 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.974953    2568 scope.go:117] "RemoveContainer" containerID="2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"
Jul 07 02:16:06.975162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975146    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309"} err="failed to get container status \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": rpc error: code = NotFound desc = could not find container \"2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309\": container with ID starting with 2d82391e8394039a0c7113b3c5abb1c7658a1cc230a7801b44aca765de886309 not found: ID does not exist"
Jul 07 02:16:06.975162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975162    2568 scope.go:117] "RemoveContainer" containerID="29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"
Jul 07 02:16:06.975338 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975320    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1"} err="failed to get container status \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": rpc error: code = NotFound desc = could not find container \"29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1\": container with ID starting with 29d2df4de1b7cdad8cf2f0e076f0eb9c7ae2512aeb1d5fcd28753286436b28c1 not found: ID does not exist"
Jul 07 02:16:06.975380 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975340    2568 scope.go:117] "RemoveContainer" containerID="38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"
Jul 07 02:16:06.975562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975547    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e"} err="failed to get container status \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": rpc error: code = NotFound desc = could not find container \"38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e\": container with ID starting with 38aaa95f11ce92f68fca6f328d172148259e1f8717ca07462b1633128bbddc7e not found: ID does not exist"
Jul 07 02:16:06.975613 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975563    2568 scope.go:117] "RemoveContainer" containerID="2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"
Jul 07 02:16:06.975782 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.975765    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b"} err="failed to get container status \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": rpc error: code = NotFound desc = could not find container \"2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b\": container with ID starting with 2c30bb72b5f5ce0bb60488f6dd3398c2e74757278c2415cae2c1bd16541eee3b not found: ID does not exist"
Jul 07 02:16:06.977226 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.977213    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:06.980210 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980188    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 02:16:06.980319 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980242    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 02:16:06.980319 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980192    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 02:16:06.980319 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980290    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-brqhh\""
Jul 07 02:16:06.980480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980187    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 02:16:06.980480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980244    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 02:16:06.980480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980262    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 02:16:06.980992 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980976    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8m6ti1p4u2q3u\""
Jul 07 02:16:06.980992 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.980985    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 02:16:06.981210 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.981192    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 02:16:06.981348 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.981330    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 02:16:06.981437 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.981346    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 02:16:06.981587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.981566    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 02:16:06.981993 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.981976    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 02:16:06.982860 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.982845    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 02:16:06.986093 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.986077    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 02:16:06.996218 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:06.996194    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:16:07.042437 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042400    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042472    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042596    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042642    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/342653ff-72c9-4626-b264-00bd3078e7b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042686    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042760    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042779    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042871 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042871 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042821    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042871 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042862    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvttq\" (UniqueName: \"kubernetes.io/projected/342653ff-72c9-4626-b264-00bd3078e7b6-kube-api-access-hvttq\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042976 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042884    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042976 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042910    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.042976 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.042941    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.043090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.043045    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.043090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.043070    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.043229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.043214    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.043294 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.043246    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.043418 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.043395    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/342653ff-72c9-4626-b264-00bd3078e7b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.043527 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.043432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-config\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144489    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144563    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144597    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144634    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144663    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144710    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144776    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/342653ff-72c9-4626-b264-00bd3078e7b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144801    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-config\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144828    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144878    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.144920 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144910    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144940    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/342653ff-72c9-4626-b264-00bd3078e7b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144966    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.144989    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145005    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145018    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145070    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145093    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145245 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145130    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvttq\" (UniqueName: \"kubernetes.io/projected/342653ff-72c9-4626-b264-00bd3078e7b6-kube-api-access-hvttq\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145440    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.145655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.145541    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.147678 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.147635    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148355    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148364    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148364    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148362    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/342653ff-72c9-4626-b264-00bd3078e7b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148574    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148859    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/342653ff-72c9-4626-b264-00bd3078e7b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.148904    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.149053    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.149221    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.149866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.149824    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.150584 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.150556    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.151206 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.151180    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.151485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.151465    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/342653ff-72c9-4626-b264-00bd3078e7b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.151558 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.151516    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.152149 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.152131    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/342653ff-72c9-4626-b264-00bd3078e7b6-config\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.155017 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.154995    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvttq\" (UniqueName: \"kubernetes.io/projected/342653ff-72c9-4626-b264-00bd3078e7b6-kube-api-access-hvttq\") pod \"prometheus-k8s-0\" (UID: \"342653ff-72c9-4626-b264-00bd3078e7b6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.288016 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.287918    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:07.429542 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.429516    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 02:16:07.430918 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:16:07.430892    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342653ff_72c9_4626_b264_00bd3078e7b6.slice/crio-e55c78f95054431520f2735f66f8c1fccb32609b92d6c45427fa0f8e82053d42 WatchSource:0}: Error finding container e55c78f95054431520f2735f66f8c1fccb32609b92d6c45427fa0f8e82053d42: Status 404 returned error can't find the container with id e55c78f95054431520f2735f66f8c1fccb32609b92d6c45427fa0f8e82053d42
Jul 07 02:16:07.919959 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.919926    2568 generic.go:356] "Generic (PLEG): container finished" podID="342653ff-72c9-4626-b264-00bd3078e7b6" containerID="3489bc9bb44bda6d393586ca6a91d63412a3d2527132a5d22b39d3ac566bb209" exitCode=0
Jul 07 02:16:07.920414 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.920011    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerDied","Data":"3489bc9bb44bda6d393586ca6a91d63412a3d2527132a5d22b39d3ac566bb209"}
Jul 07 02:16:07.920414 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:07.920096    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"e55c78f95054431520f2735f66f8c1fccb32609b92d6c45427fa0f8e82053d42"}
Jul 07 02:16:08.434123 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.434026    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c22cc3a-4ad5-412e-ba52-209a5905c981" path="/var/lib/kubelet/pods/2c22cc3a-4ad5-412e-ba52-209a5905c981/volumes"
Jul 07 02:16:08.929299 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.929254    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"f37e614fed8b6213ded78247a6f66d6c113ccc4bace88b9d94e311a06456cb35"}
Jul 07 02:16:08.929719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.929343    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"f1d72fb9fc28db24dd39611f44bfd85adebd4b5dd911d7a29d33102c75a29d3c"}
Jul 07 02:16:08.929719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.929391    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"7d8cbe40239706c60d2dff6fbc799f45b776e54528efee33955fb86a2f8b3ce6"}
Jul 07 02:16:08.929719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.929426    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"af53b6e6461ca21983f858c70f2bc7abc97b1106f8c941d1690d7c9ba0cde3fd"}
Jul 07 02:16:08.929719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.929463    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"5500fcfb672e719dbd8523245bee04ef53a363854a6fb862455bf1987af08bef"}
Jul 07 02:16:08.929719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.929515    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"342653ff-72c9-4626-b264-00bd3078e7b6","Type":"ContainerStarted","Data":"0167ec00ce8e940e6c55a5ad7a1f35e2bb481bbf0d2b1bb6a108c67989659496"}
Jul 07 02:16:08.961074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:08.961000    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.9433674930000002 podStartE2EDuration="2.960978953s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.61146ms" isStatefulPod=false podCreationTimestamp="2026-07-07 02:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:16:08.956349905 +0000 UTC m=+379.159770686" watchObservedRunningTime="2026-07-07 02:16:08.960978953 +0000 UTC m=+379.164399699"
Jul 07 02:16:09.708260 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:09.708226    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:09.765234 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:09.765199    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:12.169002 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:12.168966    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c46cb6c-qps6t"]
Jul 07 02:16:12.267699 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:12.267664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:12.288576 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:12.288551    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:16:12.316615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:12.316587    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:12.371957 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:12.371918    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:15.565899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:15.565862    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:15.666990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:15.666953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:37.189969 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.189906    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-77c46cb6c-qps6t" podUID="93cbaa45-7369-4e5b-9179-4376eeaa473e" containerName="console" containerID="cri-o://e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971" gracePeriod=15
Jul 07 02:16:37.451638 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.451572    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:16:37.504267 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504236    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-oauth-config\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.504267 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504271    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx9vn\" (UniqueName: \"kubernetes.io/projected/93cbaa45-7369-4e5b-9179-4376eeaa473e-kube-api-access-vx9vn\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.504489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504294    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-service-ca\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.504489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504325    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-oauth-serving-cert\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.504620 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504549    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-serving-cert\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.504676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504635    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-trusted-ca-bundle\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.505054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504677    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-config\") pod \"93cbaa45-7369-4e5b-9179-4376eeaa473e\" (UID: \"93cbaa45-7369-4e5b-9179-4376eeaa473e\") "
Jul 07 02:16:37.505054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504740    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:37.505054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.504747    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-service-ca" (OuterVolumeSpecName: "service-ca") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:37.505054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.505040    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:37.505306 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.505064    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-config" (OuterVolumeSpecName: "console-config") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:16:37.505306 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.505108    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-trusted-ca-bundle\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:37.505306 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.505127    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:37.505306 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.505141    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-oauth-serving-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:37.506726 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.506706    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:37.506988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.506960    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:16:37.506988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.506971    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93cbaa45-7369-4e5b-9179-4376eeaa473e-kube-api-access-vx9vn" (OuterVolumeSpecName: "kube-api-access-vx9vn") pod "93cbaa45-7369-4e5b-9179-4376eeaa473e" (UID: "93cbaa45-7369-4e5b-9179-4376eeaa473e"). InnerVolumeSpecName "kube-api-access-vx9vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:16:37.605915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.605882    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:37.605915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.605909    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-oauth-config\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:37.605915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.605919    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx9vn\" (UniqueName: \"kubernetes.io/projected/93cbaa45-7369-4e5b-9179-4376eeaa473e-kube-api-access-vx9vn\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:37.606153 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:37.605929    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cbaa45-7369-4e5b-9179-4376eeaa473e-console-serving-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:16:38.055937 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.055905    2568 generic.go:356] "Generic (PLEG): container finished" podID="93cbaa45-7369-4e5b-9179-4376eeaa473e" containerID="e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971" exitCode=0
Jul 07 02:16:38.056137 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.055949    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c46cb6c-qps6t"
Jul 07 02:16:38.056137 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.055994    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c46cb6c-qps6t" event={"ID":"93cbaa45-7369-4e5b-9179-4376eeaa473e","Type":"ContainerDied","Data":"e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971"}
Jul 07 02:16:38.056137 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.056042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c46cb6c-qps6t" event={"ID":"93cbaa45-7369-4e5b-9179-4376eeaa473e","Type":"ContainerDied","Data":"b10af68d2cef9aa2ca0fdb1de202d765e12b681561d91754da11a9b3dcc34006"}
Jul 07 02:16:38.056137 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.056064    2568 scope.go:117] "RemoveContainer" containerID="e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971"
Jul 07 02:16:38.065711 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.065692    2568 scope.go:117] "RemoveContainer" containerID="e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971"
Jul 07 02:16:38.065971 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:16:38.065951    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971\": container with ID starting with e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971 not found: ID does not exist" containerID="e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971"
Jul 07 02:16:38.066016 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.065981    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971"} err="failed to get container status \"e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971\": rpc error: code = NotFound desc = could not find container \"e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971\": container with ID starting with e51f0ee4944b80cfa97c08a32ebaebe02c3bc2b1940e59c5974831bd97e27971 not found: ID does not exist"
Jul 07 02:16:38.079095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.079064    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c46cb6c-qps6t"]
Jul 07 02:16:38.083720 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.083697    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77c46cb6c-qps6t"]
Jul 07 02:16:38.433225 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:38.433148    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93cbaa45-7369-4e5b-9179-4376eeaa473e" path="/var/lib/kubelet/pods/93cbaa45-7369-4e5b-9179-4376eeaa473e/volumes"
Jul 07 02:16:43.896237 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:43.896204    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:43.997116 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:43.997072    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:46.814521 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:46.814465    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:46.910831 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:46.910800    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:58.779387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:58.779350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:16:58.836477 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:16:58.836441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:07.288261 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:07.288216    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:17:07.303915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:07.303888    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:17:08.193769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:08.193743    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 02:17:13.974632 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:13.974538    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:14.036933 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:14.036893    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:16.236819 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:16.236788    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:21.813554 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:21.813519    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:21.914759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:21.914722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:22.703094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:22.703057    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:22.805758 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:22.805728    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:24.203344 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:24.203312    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:24.925174 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:24.925132    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:25.112951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.112920    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:25.607455 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.607421    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j926r"]
Jul 07 02:17:25.607866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.607764    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93cbaa45-7369-4e5b-9179-4376eeaa473e" containerName="console"
Jul 07 02:17:25.607866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.607775    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="93cbaa45-7369-4e5b-9179-4376eeaa473e" containerName="console"
Jul 07 02:17:25.607866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.607827    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="93cbaa45-7369-4e5b-9179-4376eeaa473e" containerName="console"
Jul 07 02:17:25.610458 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.610442    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.612917 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.612901    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 02:17:25.613768 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.613747    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 02:17:25.613768 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.613762    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dsppw\""
Jul 07 02:17:25.620212 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.620180    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j926r"]
Jul 07 02:17:25.631002 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.630975    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmv4w\" (UniqueName: \"kubernetes.io/projected/dcd20146-428f-4d56-8e02-fbba434471f4-kube-api-access-zmv4w\") pod \"cert-manager-cainjector-559cdd8c87-j926r\" (UID: \"dcd20146-428f-4d56-8e02-fbba434471f4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.631133 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.631071    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dcd20146-428f-4d56-8e02-fbba434471f4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j926r\" (UID: \"dcd20146-428f-4d56-8e02-fbba434471f4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.731859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.731823    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dcd20146-428f-4d56-8e02-fbba434471f4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j926r\" (UID: \"dcd20146-428f-4d56-8e02-fbba434471f4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.732056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.731865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmv4w\" (UniqueName: \"kubernetes.io/projected/dcd20146-428f-4d56-8e02-fbba434471f4-kube-api-access-zmv4w\") pod \"cert-manager-cainjector-559cdd8c87-j926r\" (UID: \"dcd20146-428f-4d56-8e02-fbba434471f4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.742235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.742205    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmv4w\" (UniqueName: \"kubernetes.io/projected/dcd20146-428f-4d56-8e02-fbba434471f4-kube-api-access-zmv4w\") pod \"cert-manager-cainjector-559cdd8c87-j926r\" (UID: \"dcd20146-428f-4d56-8e02-fbba434471f4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.742335 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.742288    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dcd20146-428f-4d56-8e02-fbba434471f4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j926r\" (UID: \"dcd20146-428f-4d56-8e02-fbba434471f4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:25.933822 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:25.933737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r"
Jul 07 02:17:26.059803 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:26.059624    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j926r"]
Jul 07 02:17:26.062261 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:17:26.062234    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcd20146_428f_4d56_8e02_fbba434471f4.slice/crio-71311b3fca112d84fbfb582f21a0cebd008fb0f5c9ef67caf4549c95dd880488 WatchSource:0}: Error finding container 71311b3fca112d84fbfb582f21a0cebd008fb0f5c9ef67caf4549c95dd880488: Status 404 returned error can't find the container with id 71311b3fca112d84fbfb582f21a0cebd008fb0f5c9ef67caf4549c95dd880488
Jul 07 02:17:26.253827 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:26.253782    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r" event={"ID":"dcd20146-428f-4d56-8e02-fbba434471f4","Type":"ContainerStarted","Data":"71311b3fca112d84fbfb582f21a0cebd008fb0f5c9ef67caf4549c95dd880488"}
Jul 07 02:17:28.849615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:28.849573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:28.950289 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:28.950259    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:33.123580 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:33.123538    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:34.123191 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:34.123150    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:34.188229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:34.188191    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:34.291032 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:34.290999    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r" event={"ID":"dcd20146-428f-4d56-8e02-fbba434471f4","Type":"ContainerStarted","Data":"85ab6787ac6515e03df1f799e9c01d441741d89111a29e08f195b6b60f51c425"}
Jul 07 02:17:34.307698 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:34.307653    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j926r" podStartSLOduration=2.020517556 podStartE2EDuration="9.307639465s" totalImagesPullingTime="7.287121909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:17:34.306555135 +0000 UTC m=+464.509975882" watchObservedRunningTime="2026-07-07 02:17:34.307639465 +0000 UTC m=+464.511060210"
Jul 07 02:17:37.846263 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:37.846224    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:38.057830 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:38.057796    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:44.062560 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:44.062521    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:44.166649 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:44.166614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:45.863811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:45.863779    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:46.283737 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:46.283698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:58.649141 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:58.649100    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:58.749460 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:58.749418    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:59.355155 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.355118    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:59.676701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.676604    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:59.762462 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.762431    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:17:59.774790 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.774765    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"]
Jul 07 02:17:59.779678 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.779660    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:17:59.783119 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.783096    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 02:17:59.783705 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.783690    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 02:17:59.784063 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.784047    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 02:17:59.784139 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.784123    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6ghdc\""
Jul 07 02:17:59.784236 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.784220    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 02:17:59.804928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.804895    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"]
Jul 07 02:17:59.926760 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.926665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-webhook-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:17:59.926760 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.926730    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-manifests\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:17:59.926950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.926826    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4dcc\" (UniqueName: \"kubernetes.io/projected/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-kube-api-access-l4dcc\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:17:59.926950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.926888    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-tmp\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:17:59.926950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.926920    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:17:59.932775 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:17:59.932747    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.027967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.027927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-manifests\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.028167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.027987    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4dcc\" (UniqueName: \"kubernetes.io/projected/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-kube-api-access-l4dcc\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.028167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.028021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-tmp\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.028167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.028048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.028167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.028064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-webhook-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.028415 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.028392    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-manifests\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.030544 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.030488    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-tmp\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.030659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.030619    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-webhook-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.030659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.030632    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.034184 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.034144    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.036669 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.036648    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4dcc\" (UniqueName: \"kubernetes.io/projected/fae8eb39-c5c4-49da-a675-7d7ab43e6dc2-kube-api-access-l4dcc\") pod \"opendatahub-operator-controller-manager-76797ccd4c-fgt75\" (UID: \"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.089661 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.089620    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:00.132474 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.132438    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.235457 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.235429    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.245240 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.245218    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"]
Jul 07 02:18:00.247427 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:18:00.247394    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae8eb39_c5c4_49da_a675_7d7ab43e6dc2.slice/crio-80489ff5df9b2e1cb62c5d164b900e490d47a59cb9277590aafc7641ba9d8193 WatchSource:0}: Error finding container 80489ff5df9b2e1cb62c5d164b900e490d47a59cb9277590aafc7641ba9d8193: Status 404 returned error can't find the container with id 80489ff5df9b2e1cb62c5d164b900e490d47a59cb9277590aafc7641ba9d8193
Jul 07 02:18:00.333715 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.333681    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.402702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.402669    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75" event={"ID":"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2","Type":"ContainerStarted","Data":"80489ff5df9b2e1cb62c5d164b900e490d47a59cb9277590aafc7641ba9d8193"}
Jul 07 02:18:00.433293 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.433261    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.533537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.533506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:00.641949 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:00.641912    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:04.423572 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:04.423530    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75" event={"ID":"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2","Type":"ContainerStarted","Data":"094a8ce9c5f734cf41350772a52d486f14ae9962e3f2dc52cb511afb00213b68"}
Jul 07 02:18:05.429247 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:05.429213    2568 generic.go:356] "Generic (PLEG): container finished" podID="fae8eb39-c5c4-49da-a675-7d7ab43e6dc2" containerID="094a8ce9c5f734cf41350772a52d486f14ae9962e3f2dc52cb511afb00213b68" exitCode=0
Jul 07 02:18:05.429667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:05.429276    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75" event={"ID":"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2","Type":"ContainerDied","Data":"094a8ce9c5f734cf41350772a52d486f14ae9962e3f2dc52cb511afb00213b68"}
Jul 07 02:18:06.435323 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:06.435289    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75" event={"ID":"fae8eb39-c5c4-49da-a675-7d7ab43e6dc2","Type":"ContainerStarted","Data":"b245a64d96dce24abc793f2d3ffb58a2740a7ac6e0189c93eee951b601eade49"}
Jul 07 02:18:06.435739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:06.435518    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:06.458009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:06.457960    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75" podStartSLOduration=2.755077223 podStartE2EDuration="7.457944203s" totalImagesPullingTime="3.620549026s" totalInitContainerRuntime="1.082317954s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:06.45684345 +0000 UTC m=+496.660264194" watchObservedRunningTime="2026-07-07 02:18:06.457944203 +0000 UTC m=+496.661364947"
Jul 07 02:18:10.183199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:10.183161    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:10.275974 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:10.275927    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:10.806609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:10.806569    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:10.907714 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:10.907677    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:17.441930 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:17.441898    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-fgt75"
Jul 07 02:18:18.373524 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.373476    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"]
Jul 07 02:18:18.375909 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.375888    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.378464 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.378438    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 02:18:18.378611 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.378442    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f855d\""
Jul 07 02:18:18.378611 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.378449    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:18:18.379324 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.379294    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 02:18:18.379409 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.379331    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 02:18:18.379485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.379430    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 02:18:18.386257 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.386236    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"]
Jul 07 02:18:18.488657 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.488626    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-manager-config\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.489003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.488710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-metrics-cert\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.489003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.488785    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmctf\" (UniqueName: \"kubernetes.io/projected/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-kube-api-access-tmctf\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.489003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.488809    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-cert\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.589692 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.589652    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-metrics-cert\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.589878 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.589707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmctf\" (UniqueName: \"kubernetes.io/projected/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-kube-api-access-tmctf\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.589878 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.589724    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-cert\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.589878 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.589749    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-manager-config\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.590337 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.590318    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-manager-config\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.592156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.592134    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-metrics-cert\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.592229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.592164    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-cert\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.605673 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.605645    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmctf\" (UniqueName: \"kubernetes.io/projected/fab27ad1-fa70-4ee0-aa8a-7dd561b620f2-kube-api-access-tmctf\") pod \"lws-controller-manager-5bd684c58c-fsx85\" (UID: \"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.685717 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.685627    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:18.827358 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:18.827287    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"]
Jul 07 02:18:18.829634 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:18:18.829595    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab27ad1_fa70_4ee0_aa8a_7dd561b620f2.slice/crio-fd8dde98255a60e075b100585f6b20f671fd1dee62993e8233915cfaa3987dda WatchSource:0}: Error finding container fd8dde98255a60e075b100585f6b20f671fd1dee62993e8233915cfaa3987dda: Status 404 returned error can't find the container with id fd8dde98255a60e075b100585f6b20f671fd1dee62993e8233915cfaa3987dda
Jul 07 02:18:19.068165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:19.068129    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:19.229142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:19.229109    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:19.490988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:19.490931    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85" event={"ID":"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2","Type":"ContainerStarted","Data":"fd8dde98255a60e075b100585f6b20f671fd1dee62993e8233915cfaa3987dda"}
Jul 07 02:18:19.918477 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:19.918285    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:20.018504 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:20.018466    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:20.143177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:20.143134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:20.450629 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:20.450570    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:20.626609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:20.626553    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:20.709935 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:20.709801    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:21.502669 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:21.502633    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85" event={"ID":"fab27ad1-fa70-4ee0-aa8a-7dd561b620f2","Type":"ContainerStarted","Data":"532171c5ccf12c0403c787acf0d3977ed39def79d0d075ac6d8162206d534221"}
Jul 07 02:18:21.502847 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:21.502756    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:21.518887 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:21.518847    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85" podStartSLOduration=0.95154708 podStartE2EDuration="3.518833516s" totalImagesPullingTime="2.567286436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:21.516692805 +0000 UTC m=+511.720113563" watchObservedRunningTime="2026-07-07 02:18:21.518833516 +0000 UTC m=+511.722254260"
Jul 07 02:18:29.089931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.089894    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:29.197051 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.197014    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:29.282252 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.282215    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:29.439762 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.439671    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:29.574226 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.574186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:29.689564 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.689525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:29.808468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:29.808434    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:30.097488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.097397    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:30.232186 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.232157    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"]
Jul 07 02:18:30.244110 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.244082    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.247703 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.247675    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 02:18:30.247885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.247675    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xsg9s\""
Jul 07 02:18:30.247885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.247720    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 02:18:30.247885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.247673    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 02:18:30.248201 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.248184    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 02:18:30.259091 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.259067    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"]
Jul 07 02:18:30.292805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.292769    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26a1c564-072d-42a9-95fe-d70d063b7561-tmp\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.292805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.292808    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4c99\" (UniqueName: \"kubernetes.io/projected/26a1c564-072d-42a9-95fe-d70d063b7561-kube-api-access-z4c99\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.293013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.292836    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a1c564-072d-42a9-95fe-d70d063b7561-tls-certs\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.393938 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.393847    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26a1c564-072d-42a9-95fe-d70d063b7561-tmp\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.393938 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.393886    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4c99\" (UniqueName: \"kubernetes.io/projected/26a1c564-072d-42a9-95fe-d70d063b7561-kube-api-access-z4c99\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.393938 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.393910    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a1c564-072d-42a9-95fe-d70d063b7561-tls-certs\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.394225 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:18:30.394018    2568 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 07 02:18:30.394225 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:18:30.394077    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a1c564-072d-42a9-95fe-d70d063b7561-tls-certs podName:26a1c564-072d-42a9-95fe-d70d063b7561 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:30.894060004 +0000 UTC m=+521.097480747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/26a1c564-072d-42a9-95fe-d70d063b7561-tls-certs") pod "kube-auth-proxy-94d78774d-2q5wt" (UID: "26a1c564-072d-42a9-95fe-d70d063b7561") : secret "kube-auth-proxy-tls" not found
Jul 07 02:18:30.396052 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.396025    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26a1c564-072d-42a9-95fe-d70d063b7561-tmp\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.403334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.403305    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4c99\" (UniqueName: \"kubernetes.io/projected/26a1c564-072d-42a9-95fe-d70d063b7561-kube-api-access-z4c99\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.481050 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.481005    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:30.899363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.899325    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a1c564-072d-42a9-95fe-d70d063b7561-tls-certs\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:30.901738 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:30.901719    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a1c564-072d-42a9-95fe-d70d063b7561-tls-certs\") pod \"kube-auth-proxy-94d78774d-2q5wt\" (UID: \"26a1c564-072d-42a9-95fe-d70d063b7561\") " pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:31.154817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:31.154730    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"
Jul 07 02:18:31.318867 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:31.318843    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-94d78774d-2q5wt"]
Jul 07 02:18:31.321090 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:18:31.321056    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a1c564_072d_42a9_95fe_d70d063b7561.slice/crio-b90ce96ab6813f731ab6721a47e0345470be77355d3d6f227b5a1c0b3985041e WatchSource:0}: Error finding container b90ce96ab6813f731ab6721a47e0345470be77355d3d6f227b5a1c0b3985041e: Status 404 returned error can't find the container with id b90ce96ab6813f731ab6721a47e0345470be77355d3d6f227b5a1c0b3985041e
Jul 07 02:18:31.553656 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:31.553621    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt" event={"ID":"26a1c564-072d-42a9-95fe-d70d063b7561","Type":"ContainerStarted","Data":"b90ce96ab6813f731ab6721a47e0345470be77355d3d6f227b5a1c0b3985041e"}
Jul 07 02:18:32.128019 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:32.127964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:32.243223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:32.243193    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:32.508884 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:32.508859    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-fsx85"
Jul 07 02:18:33.253219 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:33.253180    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:35.576016 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:35.575977    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt" event={"ID":"26a1c564-072d-42a9-95fe-d70d063b7561","Type":"ContainerStarted","Data":"3c07709e629c7d27db7ebcbe9f8567c5c02474823520e8be56f021c90eaf1c2f"}
Jul 07 02:18:35.594173 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:35.593968    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-94d78774d-2q5wt" podStartSLOduration=2.052459586 podStartE2EDuration="5.593945358s" totalImagesPullingTime="3.541485772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:35.593681394 +0000 UTC m=+525.797102139" watchObservedRunningTime="2026-07-07 02:18:35.593945358 +0000 UTC m=+525.797366103"
Jul 07 02:18:36.322826 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:36.322790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:36.430112 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:36.430049    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:37.000009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:36.999980    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:37.226030 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:37.225993    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:37.458096 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:37.458004    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:18:47.587318 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:18:47.587230    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:10.784488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:10.784450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:10.842541 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:10.842488    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:12.723847 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:12.723808    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:12.825422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:12.825388    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:20.595554 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:20.595489    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:20.664335 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:20.664281    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:20.775117 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:20.775086    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:33.638644 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:33.638605    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:49.725760 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:49.725720    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:50.986113 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:50.986076    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:51.350747 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:51.350706    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:52.451815 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:52.451780    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:19:52.457474 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:52.457448    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:19:54.930309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:54.930269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:57.758206 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:57.758168    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:58.369588 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:58.369541    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:19:59.663063 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.663024    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"]
Jul 07 02:19:59.665464 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.665441    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:19:59.668001 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.667973    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 02:19:59.668114 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.668016    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kd75z\""
Jul 07 02:19:59.668810 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.668793    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 02:19:59.668872 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.668808    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 02:19:59.679898 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.679875    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"]
Jul 07 02:19:59.685692 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.685669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvsbt\" (UniqueName: \"kubernetes.io/projected/b7b89f5f-bdf4-419c-b9f5-832751ca8de5-kube-api-access-lvsbt\") pod \"dns-operator-controller-manager-864dd78c4c-lzfrf\" (UID: \"b7b89f5f-bdf4-419c-b9f5-832751ca8de5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:19:59.786939 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.786908    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvsbt\" (UniqueName: \"kubernetes.io/projected/b7b89f5f-bdf4-419c-b9f5-832751ca8de5-kube-api-access-lvsbt\") pod \"dns-operator-controller-manager-864dd78c4c-lzfrf\" (UID: \"b7b89f5f-bdf4-419c-b9f5-832751ca8de5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:19:59.794983 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.794960    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvsbt\" (UniqueName: \"kubernetes.io/projected/b7b89f5f-bdf4-419c-b9f5-832751ca8de5-kube-api-access-lvsbt\") pod \"dns-operator-controller-manager-864dd78c4c-lzfrf\" (UID: \"b7b89f5f-bdf4-419c-b9f5-832751ca8de5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:19:59.976189 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:19:59.976101    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:20:00.117069 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:00.117047    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"]
Jul 07 02:20:00.120350 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:00.120328    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b89f5f_bdf4_419c_b9f5_832751ca8de5.slice/crio-6716c451db75ca808a14e4d5e1cd5454de13a3ed6dfa5c076d2c16e223276a78 WatchSource:0}: Error finding container 6716c451db75ca808a14e4d5e1cd5454de13a3ed6dfa5c076d2c16e223276a78: Status 404 returned error can't find the container with id 6716c451db75ca808a14e4d5e1cd5454de13a3ed6dfa5c076d2c16e223276a78
Jul 07 02:20:00.122560 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:00.122545    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:20:00.136887 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:00.136844    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:00.614099 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:00.614064    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:00.961768 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:00.961685    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf" event={"ID":"b7b89f5f-bdf4-419c-b9f5-832751ca8de5","Type":"ContainerStarted","Data":"6716c451db75ca808a14e4d5e1cd5454de13a3ed6dfa5c076d2c16e223276a78"}
Jul 07 02:20:02.971883 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:02.971847    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf" event={"ID":"b7b89f5f-bdf4-419c-b9f5-832751ca8de5","Type":"ContainerStarted","Data":"65e22ef3dd4a71507a28a8364893dd2cc6f37c2cace1a5c1a12665e4083615eb"}
Jul 07 02:20:02.972363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:02.971925    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:20:02.990155 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:02.990101    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf" podStartSLOduration=1.65432216 podStartE2EDuration="3.990088217s" totalImagesPullingTime="2.335766057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:02.987608761 +0000 UTC m=+613.191029507" watchObservedRunningTime="2026-07-07 02:20:02.990088217 +0000 UTC m=+613.193508960"
Jul 07 02:20:03.070517 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.070467    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d8f4bf44b-wglvc"]
Jul 07 02:20:03.073415 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.073397    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.076552 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.076530    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cnxzk\""
Jul 07 02:20:03.077485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.077459    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 02:20:03.077626 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.077574    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 02:20:03.077897 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.077817    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 02:20:03.078281 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.078119    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 02:20:03.078281 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.078188    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 02:20:03.083313 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.083296    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 02:20:03.086982 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.086959    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8f4bf44b-wglvc"]
Jul 07 02:20:03.109469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109444    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-service-ca\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.109649 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-trusted-ca-bundle\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.109649 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109560    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-config\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.109649 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-oauth-config\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.109649 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-oauth-serving-cert\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.109828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qzv5\" (UniqueName: \"kubernetes.io/projected/3a60b967-f9ad-4620-91f4-4759ebd899bd-kube-api-access-2qzv5\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.109828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.109705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-serving-cert\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.137540 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.137488    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:03.210337 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210313    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-service-ca\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.210515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210343    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-trusted-ca-bundle\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.210515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210387    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-config\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.210515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210414    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-oauth-config\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.210515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210444    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-oauth-serving-cert\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.210515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qzv5\" (UniqueName: \"kubernetes.io/projected/3a60b967-f9ad-4620-91f4-4759ebd899bd-kube-api-access-2qzv5\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.210515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.210512    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-serving-cert\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.211144 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.211123    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-service-ca\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.211270 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.211248    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-oauth-serving-cert\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.211270 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.211264    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-trusted-ca-bundle\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.211389 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.211263    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-config\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.212729 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.212708    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-oauth-config\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.212822 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.212807    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a60b967-f9ad-4620-91f4-4759ebd899bd-console-serving-cert\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.218222 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.218199    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qzv5\" (UniqueName: \"kubernetes.io/projected/3a60b967-f9ad-4620-91f4-4759ebd899bd-kube-api-access-2qzv5\") pod \"console-6d8f4bf44b-wglvc\" (UID: \"3a60b967-f9ad-4620-91f4-4759ebd899bd\") " pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.383982 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.383951    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:03.535220 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.535189    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8f4bf44b-wglvc"]
Jul 07 02:20:03.539088 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:03.539056    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a60b967_f9ad_4620_91f4_4759ebd899bd.slice/crio-0214d7c0a8c5c157c26ce389e3f946efe1e044ebd48263a82ab7cf584d946c0c WatchSource:0}: Error finding container 0214d7c0a8c5c157c26ce389e3f946efe1e044ebd48263a82ab7cf584d946c0c: Status 404 returned error can't find the container with id 0214d7c0a8c5c157c26ce389e3f946efe1e044ebd48263a82ab7cf584d946c0c
Jul 07 02:20:03.579835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.579806    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:03.977330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.977295    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8f4bf44b-wglvc" event={"ID":"3a60b967-f9ad-4620-91f4-4759ebd899bd","Type":"ContainerStarted","Data":"d498517ce6ce4686bd09501dcda87a3b1f49cad759adf0315ac0994b26d52877"}
Jul 07 02:20:03.977767 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.977348    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8f4bf44b-wglvc" event={"ID":"3a60b967-f9ad-4620-91f4-4759ebd899bd","Type":"ContainerStarted","Data":"0214d7c0a8c5c157c26ce389e3f946efe1e044ebd48263a82ab7cf584d946c0c"}
Jul 07 02:20:03.996215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:03.996172    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d8f4bf44b-wglvc" podStartSLOduration=0.996156306 podStartE2EDuration="996.156306ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:03.994831926 +0000 UTC m=+614.198252671" watchObservedRunningTime="2026-07-07 02:20:03.996156306 +0000 UTC m=+614.199577049"
Jul 07 02:20:04.053022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.052985    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"]
Jul 07 02:20:04.055312 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.055290    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.058125 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.058106    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q7vvz\""
Jul 07 02:20:04.069914 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.069892    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"]
Jul 07 02:20:04.117537 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.117490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxfc2\" (UniqueName: \"kubernetes.io/projected/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-kube-api-access-wxfc2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.117679 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.117587    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.218089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.218056    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.218234 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.218117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxfc2\" (UniqueName: \"kubernetes.io/projected/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-kube-api-access-wxfc2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.218440 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.218422    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.227929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.227870    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxfc2\" (UniqueName: \"kubernetes.io/projected/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-kube-api-access-wxfc2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.365877 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.365843    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:04.502457 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.502424    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"]
Jul 07 02:20:04.505642 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:04.505615    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0c9651_8ea5_44cc_9dd8_2d0baba9c079.slice/crio-0e8da9ece4b195df7d9290abc2f2e6512f35fecd58f2f382b6238ad251a27b56 WatchSource:0}: Error finding container 0e8da9ece4b195df7d9290abc2f2e6512f35fecd58f2f382b6238ad251a27b56: Status 404 returned error can't find the container with id 0e8da9ece4b195df7d9290abc2f2e6512f35fecd58f2f382b6238ad251a27b56
Jul 07 02:20:04.982802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:04.982719    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t" event={"ID":"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079","Type":"ContainerStarted","Data":"0e8da9ece4b195df7d9290abc2f2e6512f35fecd58f2f382b6238ad251a27b56"}
Jul 07 02:20:05.366406 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:05.366373    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:05.466326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:05.466271    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:05.741865 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:05.741824    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:05.839197 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:05.839155    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:09.144041 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:09.143999    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:09.243383 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:09.243337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:11.014972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:11.014932    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t" event={"ID":"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079","Type":"ContainerStarted","Data":"b720e1c742b7118ccbce399991aca9cde2ae697c8979c6e876119d51b9e946ad"}
Jul 07 02:20:11.015385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:11.015038    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:11.037771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:11.037719    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t" podStartSLOduration=1.210852093 podStartE2EDuration="7.037703478s" totalImagesPullingTime="5.826851385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:11.03589254 +0000 UTC m=+621.239313286" watchObservedRunningTime="2026-07-07 02:20:11.037703478 +0000 UTC m=+621.241124252"
Jul 07 02:20:12.543452 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:12.543414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:12.610489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:12.610443    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:13.005874 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:13.005840    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:13.140200 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:13.140169    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:13.384243 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:13.384143    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:13.384243 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:13.384206    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:13.389187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:13.389165    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:13.980164 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:13.980134    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lzfrf"
Jul 07 02:20:14.033987 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:14.033958    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d8f4bf44b-wglvc"
Jul 07 02:20:14.322605 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:14.322568    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:14.499919 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:14.499885    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:14.620433 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:14.620317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:16.670156 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:16.670113    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:16.762380 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:16.762343    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:22.021965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:22.021938    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:23.171557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.171521    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"]
Jul 07 02:20:23.178176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.178147    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.191950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.191918    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"]
Jul 07 02:20:23.295653 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.295619    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19aaf2f3-de14-41c0-a361-0cf54e214ba1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.295834 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.295662    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvh7\" (UniqueName: \"kubernetes.io/projected/19aaf2f3-de14-41c0-a361-0cf54e214ba1-kube-api-access-gjvh7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.396446 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.396405    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19aaf2f3-de14-41c0-a361-0cf54e214ba1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.396655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.396461    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvh7\" (UniqueName: \"kubernetes.io/projected/19aaf2f3-de14-41c0-a361-0cf54e214ba1-kube-api-access-gjvh7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.396826 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.396806    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19aaf2f3-de14-41c0-a361-0cf54e214ba1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.407730 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.407706    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvh7\" (UniqueName: \"kubernetes.io/projected/19aaf2f3-de14-41c0-a361-0cf54e214ba1-kube-api-access-gjvh7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.488809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.488730    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:23.622722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.622658    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"]
Jul 07 02:20:23.625121 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:23.625087    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19aaf2f3_de14_41c0_a361_0cf54e214ba1.slice/crio-d970d911d324e027dbfbf14358e7e8fb2f3ac37185320178aad67535d35ac0f1 WatchSource:0}: Error finding container d970d911d324e027dbfbf14358e7e8fb2f3ac37185320178aad67535d35ac0f1: Status 404 returned error can't find the container with id d970d911d324e027dbfbf14358e7e8fb2f3ac37185320178aad67535d35ac0f1
Jul 07 02:20:23.826750 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.826714    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"]
Jul 07 02:20:23.840166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.840136    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"]
Jul 07 02:20:23.858868 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.858838    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"]
Jul 07 02:20:23.859084 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.859039    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" containerName="manager" containerID="cri-o://b720e1c742b7118ccbce399991aca9cde2ae697c8979c6e876119d51b9e946ad" gracePeriod=2
Jul 07 02:20:23.861393 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.861367    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"]
Jul 07 02:20:23.861932 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.861909    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" containerName="manager"
Jul 07 02:20:23.862438 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.862416    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" containerName="manager"
Jul 07 02:20:23.862621 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.862604    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" containerName="manager"
Jul 07 02:20:23.865805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.865788    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:23.866293 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.866145    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"]
Jul 07 02:20:23.869841 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:23.869812    2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-6k67t\": pods \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:23.877797 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.876553    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"]
Jul 07 02:20:23.880174 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.880148    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"]
Jul 07 02:20:23.880981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.880957    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" containerName="manager"
Jul 07 02:20:23.880981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.880980    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" containerName="manager"
Jul 07 02:20:23.881148 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.881124    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" containerName="manager"
Jul 07 02:20:23.884222 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.884202    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:23.898016 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:23.897995    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"]
Jul 07 02:20:23.903377 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:23.903350    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:23.929584 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:23.929553    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:24.002227 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.002183    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b521bf23-6be5-4c87-8b5a-b901a20afce1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-657hm\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.002384 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.002247    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgnxc\" (UniqueName: \"kubernetes.io/projected/fb79e44d-83c4-4506-9cb0-e979bd881366-kube-api-access-bgnxc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dwlcv\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.002426 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.002388    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwr4w\" (UniqueName: \"kubernetes.io/projected/b521bf23-6be5-4c87-8b5a-b901a20afce1-kube-api-access-hwr4w\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-657hm\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.002466 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.002449    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb79e44d-83c4-4506-9cb0-e979bd881366-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dwlcv\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.082568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.082478    2568 generic.go:356] "Generic (PLEG): container finished" podID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" containerID="b720e1c742b7118ccbce399991aca9cde2ae697c8979c6e876119d51b9e946ad" exitCode=0
Jul 07 02:20:24.082568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.082564    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e8da9ece4b195df7d9290abc2f2e6512f35fecd58f2f382b6238ad251a27b56"
Jul 07 02:20:24.084006 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.083987    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:24.084304 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:24.084290    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8" container="manager"
Jul 07 02:20:24.086210 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:24.086189    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:24.087974 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:24.087953    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6k67t\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:24.090071 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:24.090052    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:24.103645 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.103625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b521bf23-6be5-4c87-8b5a-b901a20afce1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-657hm\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.103701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.103671    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgnxc\" (UniqueName: \"kubernetes.io/projected/fb79e44d-83c4-4506-9cb0-e979bd881366-kube-api-access-bgnxc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dwlcv\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.103755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.103721    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwr4w\" (UniqueName: \"kubernetes.io/projected/b521bf23-6be5-4c87-8b5a-b901a20afce1-kube-api-access-hwr4w\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-657hm\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.103809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.103756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb79e44d-83c4-4506-9cb0-e979bd881366-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dwlcv\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.103972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.103952    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b521bf23-6be5-4c87-8b5a-b901a20afce1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-657hm\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.104151 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.104134    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb79e44d-83c4-4506-9cb0-e979bd881366-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dwlcv\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.110970 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.110943    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:24.112726 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.112705    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgnxc\" (UniqueName: \"kubernetes.io/projected/fb79e44d-83c4-4506-9cb0-e979bd881366-kube-api-access-bgnxc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dwlcv\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.113003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.112988    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwr4w\" (UniqueName: \"kubernetes.io/projected/b521bf23-6be5-4c87-8b5a-b901a20afce1-kube-api-access-hwr4w\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-657hm\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.205074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.205044    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-extensions-socket-volume\") pod \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") "
Jul 07 02:20:24.205543 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.205095    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxfc2\" (UniqueName: \"kubernetes.io/projected/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-kube-api-access-wxfc2\") pod \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\" (UID: \"6a0c9651-8ea5-44cc-9dd8-2d0baba9c079\") "
Jul 07 02:20:24.205659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.205638    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" (UID: "6a0c9651-8ea5-44cc-9dd8-2d0baba9c079"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:20:24.207246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.207229    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-kube-api-access-wxfc2" (OuterVolumeSpecName: "kube-api-access-wxfc2") pod "6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" (UID: "6a0c9651-8ea5-44cc-9dd8-2d0baba9c079"). InnerVolumeSpecName "kube-api-access-wxfc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:20:24.213246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.213224    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:24.224004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.223988    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:24.230968 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.230951    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:24.306216 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.306186    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-extensions-socket-volume\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:24.306353 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.306220    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxfc2\" (UniqueName: \"kubernetes.io/projected/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079-kube-api-access-wxfc2\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:24.376756 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.376731    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"]
Jul 07 02:20:24.379891 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:24.379864    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb521bf23_6be5_4c87_8b5a_b901a20afce1.slice/crio-37ab0de9f5669976477bb0e53845e62212d3b4fd51c951445ad909d3900c6389 WatchSource:0}: Error finding container 37ab0de9f5669976477bb0e53845e62212d3b4fd51c951445ad909d3900c6389: Status 404 returned error can't find the container with id 37ab0de9f5669976477bb0e53845e62212d3b4fd51c951445ad909d3900c6389
Jul 07 02:20:24.392802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.392739    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"]
Jul 07 02:20:24.394969 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:24.394945    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb79e44d_83c4_4506_9cb0_e979bd881366.slice/crio-3ab99841eedff88b8488135dc4febc2760db5a1efaca9a7b62dcb09cedc89b69 WatchSource:0}: Error finding container 3ab99841eedff88b8488135dc4febc2760db5a1efaca9a7b62dcb09cedc89b69: Status 404 returned error can't find the container with id 3ab99841eedff88b8488135dc4febc2760db5a1efaca9a7b62dcb09cedc89b69
Jul 07 02:20:24.435410 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.435382    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a0c9651-8ea5-44cc-9dd8-2d0baba9c079" path="/var/lib/kubelet/pods/6a0c9651-8ea5-44cc-9dd8-2d0baba9c079/volumes"
Jul 07 02:20:24.471071 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.471032    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:24.566928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:24.566889    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:25.057107 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.057057    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:25.090795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.090755    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm" event={"ID":"b521bf23-6be5-4c87-8b5a-b901a20afce1","Type":"ContainerStarted","Data":"406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1"}
Jul 07 02:20:25.090988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.090858    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:25.090988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.090886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm" event={"ID":"b521bf23-6be5-4c87-8b5a-b901a20afce1","Type":"ContainerStarted","Data":"37ab0de9f5669976477bb0e53845e62212d3b4fd51c951445ad909d3900c6389"}
Jul 07 02:20:25.092408 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.092374    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" containerName="manager" containerID="cri-o://1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c" gracePeriod=2
Jul 07 02:20:25.094099 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.094068    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv" event={"ID":"fb79e44d-83c4-4506-9cb0-e979bd881366","Type":"ContainerStarted","Data":"644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b"}
Jul 07 02:20:25.094209 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.094122    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:25.094209 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.094139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv" event={"ID":"fb79e44d-83c4-4506-9cb0-e979bd881366","Type":"ContainerStarted","Data":"3ab99841eedff88b8488135dc4febc2760db5a1efaca9a7b62dcb09cedc89b69"}
Jul 07 02:20:25.094209 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.094075    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6k67t"
Jul 07 02:20:25.111279 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.111232    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm" podStartSLOduration=2.11121483 podStartE2EDuration="2.11121483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:25.110327703 +0000 UTC m=+635.313748437" watchObservedRunningTime="2026-07-07 02:20:25.11121483 +0000 UTC m=+635.314635576"
Jul 07 02:20:25.132101 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.132056    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv" podStartSLOduration=2.132041427 podStartE2EDuration="2.132041427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:25.130732473 +0000 UTC m=+635.334153214" watchObservedRunningTime="2026-07-07 02:20:25.132041427 +0000 UTC m=+635.335462171"
Jul 07 02:20:25.137032 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.137007    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:25.342290 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.342266    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:25.345188 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:25.345163    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:25.417275 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.417241    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19aaf2f3-de14-41c0-a361-0cf54e214ba1-extensions-socket-volume\") pod \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") "
Jul 07 02:20:25.417430 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.417359    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjvh7\" (UniqueName: \"kubernetes.io/projected/19aaf2f3-de14-41c0-a361-0cf54e214ba1-kube-api-access-gjvh7\") pod \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\" (UID: \"19aaf2f3-de14-41c0-a361-0cf54e214ba1\") "
Jul 07 02:20:25.417543 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.417518    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19aaf2f3-de14-41c0-a361-0cf54e214ba1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "19aaf2f3-de14-41c0-a361-0cf54e214ba1" (UID: "19aaf2f3-de14-41c0-a361-0cf54e214ba1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:20:25.419367 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.419343    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19aaf2f3-de14-41c0-a361-0cf54e214ba1-kube-api-access-gjvh7" (OuterVolumeSpecName: "kube-api-access-gjvh7") pod "19aaf2f3-de14-41c0-a361-0cf54e214ba1" (UID: "19aaf2f3-de14-41c0-a361-0cf54e214ba1"). InnerVolumeSpecName "kube-api-access-gjvh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:20:25.518192 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.518153    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19aaf2f3-de14-41c0-a361-0cf54e214ba1-extensions-socket-volume\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:25.518192 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:25.518185    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjvh7\" (UniqueName: \"kubernetes.io/projected/19aaf2f3-de14-41c0-a361-0cf54e214ba1-kube-api-access-gjvh7\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:26.100078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:26.100041    2568 generic.go:356] "Generic (PLEG): container finished" podID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" containerID="1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c" exitCode=2
Jul 07 02:20:26.100078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:26.100082    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:26.100324 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:26.100136    2568 scope.go:117] "RemoveContainer" containerID="1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c"
Jul 07 02:20:26.102736 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:26.102703    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:26.109605 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:26.109589    2568 scope.go:117] "RemoveContainer" containerID="1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c"
Jul 07 02:20:26.109897 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:26.109879    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c\": container with ID starting with 1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c not found: ID does not exist" containerID="1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c"
Jul 07 02:20:26.109947 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:26.109906    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c"} err="failed to get container status \"1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c\": rpc error: code = NotFound desc = could not find container \"1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c\": container with ID starting with 1172e5be9eba6e18fb27e2720b3f268b8f171145160e1f553893fc8139be951c not found: ID does not exist"
Jul 07 02:20:26.110583 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:26.110563    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-jpmn8\" is forbidden: User \"system:node:ip-10-0-143-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-152.ec2.internal' and this object" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jpmn8"
Jul 07 02:20:26.432844 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:26.432766    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19aaf2f3-de14-41c0-a361-0cf54e214ba1" path="/var/lib/kubelet/pods/19aaf2f3-de14-41c0-a361-0cf54e214ba1/volumes"
Jul 07 02:20:28.437601 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:28.437571    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:28.537916 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:28.537887    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:30.840341 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:30.840300    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:30.942967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:30.942936    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:35.419835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:35.419798    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:35.518203 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:35.518167    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:36.103877 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.103845    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:36.104762 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.104739    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:36.182991 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.182954    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"]
Jul 07 02:20:36.183236 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.183208    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv" podUID="fb79e44d-83c4-4506-9cb0-e979bd881366" containerName="manager" containerID="cri-o://644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b" gracePeriod=10
Jul 07 02:20:36.437217 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.437194    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:36.518720 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.518691    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgnxc\" (UniqueName: \"kubernetes.io/projected/fb79e44d-83c4-4506-9cb0-e979bd881366-kube-api-access-bgnxc\") pod \"fb79e44d-83c4-4506-9cb0-e979bd881366\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") "
Jul 07 02:20:36.518880 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.518793    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb79e44d-83c4-4506-9cb0-e979bd881366-extensions-socket-volume\") pod \"fb79e44d-83c4-4506-9cb0-e979bd881366\" (UID: \"fb79e44d-83c4-4506-9cb0-e979bd881366\") "
Jul 07 02:20:36.519175 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.519155    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb79e44d-83c4-4506-9cb0-e979bd881366-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb79e44d-83c4-4506-9cb0-e979bd881366" (UID: "fb79e44d-83c4-4506-9cb0-e979bd881366"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:20:36.520643 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.520618    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb79e44d-83c4-4506-9cb0-e979bd881366-kube-api-access-bgnxc" (OuterVolumeSpecName: "kube-api-access-bgnxc") pod "fb79e44d-83c4-4506-9cb0-e979bd881366" (UID: "fb79e44d-83c4-4506-9cb0-e979bd881366"). InnerVolumeSpecName "kube-api-access-bgnxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:20:36.557607 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.557573    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"]
Jul 07 02:20:36.557944 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.557929    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb79e44d-83c4-4506-9cb0-e979bd881366" containerName="manager"
Jul 07 02:20:36.558047 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.557945    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb79e44d-83c4-4506-9cb0-e979bd881366" containerName="manager"
Jul 07 02:20:36.558047 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.558021    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb79e44d-83c4-4506-9cb0-e979bd881366" containerName="manager"
Jul 07 02:20:36.560005 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.559987    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.576471 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.576448    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"]
Jul 07 02:20:36.619968 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.619938    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb79e44d-83c4-4506-9cb0-e979bd881366-extensions-socket-volume\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:36.619968 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.619964    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgnxc\" (UniqueName: \"kubernetes.io/projected/fb79e44d-83c4-4506-9cb0-e979bd881366-kube-api-access-bgnxc\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:36.721310 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.721215    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tzk4m\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.721310 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.721275    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp4zm\" (UniqueName: \"kubernetes.io/projected/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-kube-api-access-zp4zm\") pod \"kuadrant-operator-controller-manager-86878c769f-tzk4m\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.822282 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.822247    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tzk4m\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.822426 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.822301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp4zm\" (UniqueName: \"kubernetes.io/projected/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-kube-api-access-zp4zm\") pod \"kuadrant-operator-controller-manager-86878c769f-tzk4m\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.822663 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.822641    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tzk4m\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.830616 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.830587    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp4zm\" (UniqueName: \"kubernetes.io/projected/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-kube-api-access-zp4zm\") pod \"kuadrant-operator-controller-manager-86878c769f-tzk4m\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:36.869626 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:36.869595    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:37.006850 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.006824    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"]
Jul 07 02:20:37.008856 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:20:37.008832    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d9707a_3eca_4cf7_b676_d1b2c366a8df.slice/crio-9d4685ea4995e0c598030ba843fc532cc91fd671a114025faf44859785500109 WatchSource:0}: Error finding container 9d4685ea4995e0c598030ba843fc532cc91fd671a114025faf44859785500109: Status 404 returned error can't find the container with id 9d4685ea4995e0c598030ba843fc532cc91fd671a114025faf44859785500109
Jul 07 02:20:37.161881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.161841    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m" event={"ID":"e3d9707a-3eca-4cf7-b676-d1b2c366a8df","Type":"ContainerStarted","Data":"bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798"}
Jul 07 02:20:37.162093 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.161910    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m" event={"ID":"e3d9707a-3eca-4cf7-b676-d1b2c366a8df","Type":"ContainerStarted","Data":"9d4685ea4995e0c598030ba843fc532cc91fd671a114025faf44859785500109"}
Jul 07 02:20:37.162093 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.161941    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:37.163355 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.163330    2568 generic.go:356] "Generic (PLEG): container finished" podID="fb79e44d-83c4-4506-9cb0-e979bd881366" containerID="644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b" exitCode=0
Jul 07 02:20:37.163488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.163367    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"
Jul 07 02:20:37.163488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.163395    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv" event={"ID":"fb79e44d-83c4-4506-9cb0-e979bd881366","Type":"ContainerDied","Data":"644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b"}
Jul 07 02:20:37.163488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.163430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv" event={"ID":"fb79e44d-83c4-4506-9cb0-e979bd881366","Type":"ContainerDied","Data":"3ab99841eedff88b8488135dc4febc2760db5a1efaca9a7b62dcb09cedc89b69"}
Jul 07 02:20:37.163488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.163450    2568 scope.go:117] "RemoveContainer" containerID="644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b"
Jul 07 02:20:37.174983 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.173317    2568 scope.go:117] "RemoveContainer" containerID="644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b"
Jul 07 02:20:37.175412 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:37.175386    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b\": container with ID starting with 644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b not found: ID does not exist" containerID="644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b"
Jul 07 02:20:37.175548 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.175426    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b"} err="failed to get container status \"644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b\": rpc error: code = NotFound desc = could not find container \"644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b\": container with ID starting with 644359d2bbde1916cced83781e129071dd292850b31cce3a45dd327d0afa762b not found: ID does not exist"
Jul 07 02:20:37.191237 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.191182    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m" podStartSLOduration=1.19116685 podStartE2EDuration="1.19116685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:37.19006943 +0000 UTC m=+647.393490175" watchObservedRunningTime="2026-07-07 02:20:37.19116685 +0000 UTC m=+647.394587641"
Jul 07 02:20:37.208795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.208770    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"]
Jul 07 02:20:37.222799 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:37.218892    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dwlcv"]
Jul 07 02:20:38.433478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:38.433443    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb79e44d-83c4-4506-9cb0-e979bd881366" path="/var/lib/kubelet/pods/fb79e44d-83c4-4506-9cb0-e979bd881366/volumes"
Jul 07 02:20:48.172164 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.172129    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:20:48.222632 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.222598    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"]
Jul 07 02:20:48.222912 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.222888    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm" podUID="b521bf23-6be5-4c87-8b5a-b901a20afce1" containerName="manager" containerID="cri-o://406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1" gracePeriod=10
Jul 07 02:20:48.476057 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.476029    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:48.523934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.523903    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwr4w\" (UniqueName: \"kubernetes.io/projected/b521bf23-6be5-4c87-8b5a-b901a20afce1-kube-api-access-hwr4w\") pod \"b521bf23-6be5-4c87-8b5a-b901a20afce1\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") "
Jul 07 02:20:48.524049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.524013    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b521bf23-6be5-4c87-8b5a-b901a20afce1-extensions-socket-volume\") pod \"b521bf23-6be5-4c87-8b5a-b901a20afce1\" (UID: \"b521bf23-6be5-4c87-8b5a-b901a20afce1\") "
Jul 07 02:20:48.524379 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.524349    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b521bf23-6be5-4c87-8b5a-b901a20afce1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b521bf23-6be5-4c87-8b5a-b901a20afce1" (UID: "b521bf23-6be5-4c87-8b5a-b901a20afce1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:20:48.525854 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.525830    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b521bf23-6be5-4c87-8b5a-b901a20afce1-kube-api-access-hwr4w" (OuterVolumeSpecName: "kube-api-access-hwr4w") pod "b521bf23-6be5-4c87-8b5a-b901a20afce1" (UID: "b521bf23-6be5-4c87-8b5a-b901a20afce1"). InnerVolumeSpecName "kube-api-access-hwr4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:20:48.624923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.624890    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b521bf23-6be5-4c87-8b5a-b901a20afce1-extensions-socket-volume\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:48.624923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:48.624920    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwr4w\" (UniqueName: \"kubernetes.io/projected/b521bf23-6be5-4c87-8b5a-b901a20afce1-kube-api-access-hwr4w\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:20:49.229901 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.229867    2568 generic.go:356] "Generic (PLEG): container finished" podID="b521bf23-6be5-4c87-8b5a-b901a20afce1" containerID="406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1" exitCode=0
Jul 07 02:20:49.230264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.229922    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"
Jul 07 02:20:49.230264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.229961    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm" event={"ID":"b521bf23-6be5-4c87-8b5a-b901a20afce1","Type":"ContainerDied","Data":"406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1"}
Jul 07 02:20:49.230264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.230012    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm" event={"ID":"b521bf23-6be5-4c87-8b5a-b901a20afce1","Type":"ContainerDied","Data":"37ab0de9f5669976477bb0e53845e62212d3b4fd51c951445ad909d3900c6389"}
Jul 07 02:20:49.230264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.230036    2568 scope.go:117] "RemoveContainer" containerID="406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1"
Jul 07 02:20:49.238351 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.238336    2568 scope.go:117] "RemoveContainer" containerID="406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1"
Jul 07 02:20:49.238619 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:20:49.238600    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1\": container with ID starting with 406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1 not found: ID does not exist" containerID="406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1"
Jul 07 02:20:49.238676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.238628    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1"} err="failed to get container status \"406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1\": rpc error: code = NotFound desc = could not find container \"406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1\": container with ID starting with 406ab192e1fba2934795aa25975e4784477dd273a3fd865014264318bcc130a1 not found: ID does not exist"
Jul 07 02:20:49.253759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.253740    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"]
Jul 07 02:20:49.266649 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:49.266623    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-657hm"]
Jul 07 02:20:50.433302 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:50.433266    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b521bf23-6be5-4c87-8b5a-b901a20afce1" path="/var/lib/kubelet/pods/b521bf23-6be5-4c87-8b5a-b901a20afce1/volumes"
Jul 07 02:20:52.555679 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:52.555643    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:20:52.725987 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:20:52.725946    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:07.911967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:07.911912    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:08.024936 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.024900    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:08.121686 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.121639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:08.345786 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.345747    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:21:08.346252 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.346225    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b521bf23-6be5-4c87-8b5a-b901a20afce1" containerName="manager"
Jul 07 02:21:08.346252 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.346245    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b521bf23-6be5-4c87-8b5a-b901a20afce1" containerName="manager"
Jul 07 02:21:08.346414 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.346362    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b521bf23-6be5-4c87-8b5a-b901a20afce1" containerName="manager"
Jul 07 02:21:08.349706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.349683    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.352332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.352311    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 02:21:08.352452 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.352317    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2gnbx\""
Jul 07 02:21:08.358073 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.358052    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:21:08.387560 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.387531    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:21:08.396307 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.396286    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a72452b3-1a89-4a92-af84-b4f97071b2d8-config-file\") pod \"limitador-limitador-54d6d988d5-d9lpl\" (UID: \"a72452b3-1a89-4a92-af84-b4f97071b2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.396449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.396361    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfbdf\" (UniqueName: \"kubernetes.io/projected/a72452b3-1a89-4a92-af84-b4f97071b2d8-kube-api-access-bfbdf\") pod \"limitador-limitador-54d6d988d5-d9lpl\" (UID: \"a72452b3-1a89-4a92-af84-b4f97071b2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.497842 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.497805    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbdf\" (UniqueName: \"kubernetes.io/projected/a72452b3-1a89-4a92-af84-b4f97071b2d8-kube-api-access-bfbdf\") pod \"limitador-limitador-54d6d988d5-d9lpl\" (UID: \"a72452b3-1a89-4a92-af84-b4f97071b2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.498032 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.497873    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a72452b3-1a89-4a92-af84-b4f97071b2d8-config-file\") pod \"limitador-limitador-54d6d988d5-d9lpl\" (UID: \"a72452b3-1a89-4a92-af84-b4f97071b2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.498543 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.498525    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a72452b3-1a89-4a92-af84-b4f97071b2d8-config-file\") pod \"limitador-limitador-54d6d988d5-d9lpl\" (UID: \"a72452b3-1a89-4a92-af84-b4f97071b2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.505862 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.505839    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfbdf\" (UniqueName: \"kubernetes.io/projected/a72452b3-1a89-4a92-af84-b4f97071b2d8-kube-api-access-bfbdf\") pod \"limitador-limitador-54d6d988d5-d9lpl\" (UID: \"a72452b3-1a89-4a92-af84-b4f97071b2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.661030 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.660941    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:08.782793 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.782755    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:08.811411 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.811388    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:21:08.813698 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:21:08.813665    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72452b3_1a89_4a92_af84_b4f97071b2d8.slice/crio-b525c4f13d437e8ec0851d7b0754045eea440797e08eaf549b40434d602f9b95 WatchSource:0}: Error finding container b525c4f13d437e8ec0851d7b0754045eea440797e08eaf549b40434d602f9b95: Status 404 returned error can't find the container with id b525c4f13d437e8ec0851d7b0754045eea440797e08eaf549b40434d602f9b95
Jul 07 02:21:08.853186 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:08.853146    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:09.330425 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:09.330387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl" event={"ID":"a72452b3-1a89-4a92-af84-b4f97071b2d8","Type":"ContainerStarted","Data":"b525c4f13d437e8ec0851d7b0754045eea440797e08eaf549b40434d602f9b95"}
Jul 07 02:21:12.349369 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:12.349331    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl" event={"ID":"a72452b3-1a89-4a92-af84-b4f97071b2d8","Type":"ContainerStarted","Data":"a508d23373904cc6ed45af220bcfa03ae3932d70b41a61b44bb03e7f96c84725"}
Jul 07 02:21:12.349787 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:12.349424    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:12.368531 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:12.368460    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl" podStartSLOduration=1.741793851 podStartE2EDuration="4.368445797s" totalImagesPullingTime="2.626651946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:12.366733246 +0000 UTC m=+682.570153991" watchObservedRunningTime="2026-07-07 02:21:12.368445797 +0000 UTC m=+682.571866573"
Jul 07 02:21:23.355379 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:23.355349    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"
Jul 07 02:21:26.441095 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:26.441058    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:39.137949 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.137911    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:39.314708 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.314679    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:39.638683 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.638648    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-955598c66-jvhqt"]
Jul 07 02:21:39.642166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.642141    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.645683 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.645660    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 02:21:39.645827 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.645682    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7cf92\""
Jul 07 02:21:39.652129 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.652107    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-955598c66-jvhqt"]
Jul 07 02:21:39.692629 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.692594    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj2ww\" (UniqueName: \"kubernetes.io/projected/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-kube-api-access-rj2ww\") pod \"authorino-955598c66-jvhqt\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") " pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.692808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.692690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-tls-cert\") pod \"authorino-955598c66-jvhqt\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") " pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.793289 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.793253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-tls-cert\") pod \"authorino-955598c66-jvhqt\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") " pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.793467 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.793326    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj2ww\" (UniqueName: \"kubernetes.io/projected/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-kube-api-access-rj2ww\") pod \"authorino-955598c66-jvhqt\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") " pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.795708 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.795689    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-tls-cert\") pod \"authorino-955598c66-jvhqt\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") " pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.801098 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.801079    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj2ww\" (UniqueName: \"kubernetes.io/projected/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-kube-api-access-rj2ww\") pod \"authorino-955598c66-jvhqt\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") " pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:39.951691 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:39.951597    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:21:40.082875 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:40.082851    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-955598c66-jvhqt"]
Jul 07 02:21:40.085602 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:21:40.085572    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15860fa8_b5c0_4a1e_8c08_ff53afdd6e21.slice/crio-f3d79f511c54194df3ad47c2274a64cdef92e56868ef37fec00cb36d41b76698 WatchSource:0}: Error finding container f3d79f511c54194df3ad47c2274a64cdef92e56868ef37fec00cb36d41b76698: Status 404 returned error can't find the container with id f3d79f511c54194df3ad47c2274a64cdef92e56868ef37fec00cb36d41b76698
Jul 07 02:21:40.487215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:40.487171    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-955598c66-jvhqt" event={"ID":"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21","Type":"ContainerStarted","Data":"f3d79f511c54194df3ad47c2274a64cdef92e56868ef37fec00cb36d41b76698"}
Jul 07 02:21:43.505568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:43.505536    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-955598c66-jvhqt" event={"ID":"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21","Type":"ContainerStarted","Data":"3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52"}
Jul 07 02:21:43.525964 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:43.525911    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-955598c66-jvhqt" podStartSLOduration=1.643307564 podStartE2EDuration="4.525895919s" totalImagesPullingTime="2.882588355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:43.524433907 +0000 UTC m=+713.727854664" watchObservedRunningTime="2026-07-07 02:21:43.525895919 +0000 UTC m=+713.729316663"
Jul 07 02:21:47.869129 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:47.869082    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:47.968381 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:47.968341    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:48.063948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:48.063910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:48.120946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:48.120857    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:50.537535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:50.537458    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:21:50.650615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:21:50.650575    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:21.059878 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:21.059845    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:21.536207 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:21.536155    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:21.984434 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:21.984345    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:22.379035 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:22.378998    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:22.492385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:22.492338    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:22.653090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:22.653014    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:22:28.010688 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:28.010639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:28.137534 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:28.137489    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:32.217842 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:32.217806    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:32.374830 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:32.374772    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:50.681410 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:50.681377    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:51.101140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:51.101105    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:22:57.713264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:57.713224    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:22:58.483545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:22:58.483508    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:23:10.380637 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:10.380601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:23:11.443017 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:11.442981    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:23:30.439818 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:30.439777    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:23:31.762579 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:31.762544    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:23:35.289123 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:35.289089    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:23:37.568578 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:37.568540    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:23:45.247088 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.247053    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"]
Jul 07 02:23:45.250323 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.250308    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.255582 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.255559    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 02:23:45.255717 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.255584    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 02:23:45.255717 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.255663    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 02:23:45.256407 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.256389    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bzvlw\""
Jul 07 02:23:45.277375 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.277351    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"]
Jul 07 02:23:45.347433 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.347398    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd9b3005-cbb9-4057-9046-20a8588346a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.347671 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.347542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.347671 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.347572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkhxx\" (UniqueName: \"kubernetes.io/projected/bd9b3005-cbb9-4057-9046-20a8588346a3-kube-api-access-dkhxx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.347671 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.347617    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.347671 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.347662    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.347845 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.347687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.412010 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.411964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:23:45.448690 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.448666    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.448795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.448697    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkhxx\" (UniqueName: \"kubernetes.io/projected/bd9b3005-cbb9-4057-9046-20a8588346a3-kube-api-access-dkhxx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.448795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.448733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.448795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.448763    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.448795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.448784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.449004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.448812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd9b3005-cbb9-4057-9046-20a8588346a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.449067 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.449048    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.449122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.449102    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.449279 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.449253    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.450980 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.450959    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd9b3005-cbb9-4057-9046-20a8588346a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.451330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.451313    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd9b3005-cbb9-4057-9046-20a8588346a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.459309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.459277    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkhxx\" (UniqueName: \"kubernetes.io/projected/bd9b3005-cbb9-4057-9046-20a8588346a3-kube-api-access-dkhxx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz\" (UID: \"bd9b3005-cbb9-4057-9046-20a8588346a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.559684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.559588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:45.710345 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:45.710314    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"]
Jul 07 02:23:45.713727 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:23:45.713694    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd9b3005_cbb9_4057_9046_20a8588346a3.slice/crio-d3e600b5a7c2ae7396d5fea20bd96f8f6c3bf91a6d3a1d19bbee331a0ad3db2a WatchSource:0}: Error finding container d3e600b5a7c2ae7396d5fea20bd96f8f6c3bf91a6d3a1d19bbee331a0ad3db2a: Status 404 returned error can't find the container with id d3e600b5a7c2ae7396d5fea20bd96f8f6c3bf91a6d3a1d19bbee331a0ad3db2a
Jul 07 02:23:46.116633 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:46.116594    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz" event={"ID":"bd9b3005-cbb9-4057-9046-20a8588346a3","Type":"ContainerStarted","Data":"d3e600b5a7c2ae7396d5fea20bd96f8f6c3bf91a6d3a1d19bbee331a0ad3db2a"}
Jul 07 02:23:47.123224 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:47.123138    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz" event={"ID":"bd9b3005-cbb9-4057-9046-20a8588346a3","Type":"ContainerStarted","Data":"e6ab81984dd35ddc9f866ce92bcf86b970dd867da67b072812f6d8a4eb32ba24"}
Jul 07 02:23:47.150298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:47.150246    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz" podStartSLOduration=1.113681324 podStartE2EDuration="2.150230062s" totalImagesPullingTime="1.036548738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:23:47.148892035 +0000 UTC m=+837.352312778" watchObservedRunningTime="2026-07-07 02:23:47.150230062 +0000 UTC m=+837.353650806"
Jul 07 02:23:48.162118 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:48.162084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:23:55.560622 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:55.560577    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:55.561078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:55.560879    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:55.576574 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:55.576552    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:56.179068 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:56.179041    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz"
Jul 07 02:23:59.463679 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:23:59.463646    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:10.181417 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:10.181379    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:10.297841 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:10.297794    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:29.906181 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:29.906145    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:30.074180 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:30.074138    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:30.869606 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:30.869570    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-955598c66-jvhqt"]
Jul 07 02:24:30.869804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:30.869756    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-955598c66-jvhqt" podUID="15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" containerName="authorino" containerID="cri-o://3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52" gracePeriod=30
Jul 07 02:24:31.119330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.119309    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:24:31.159140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.159007    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:31.165943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.165919    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj2ww\" (UniqueName: \"kubernetes.io/projected/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-kube-api-access-rj2ww\") pod \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") "
Jul 07 02:24:31.166096 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.165985    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-tls-cert\") pod \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\" (UID: \"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21\") "
Jul 07 02:24:31.168223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.168189    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-kube-api-access-rj2ww" (OuterVolumeSpecName: "kube-api-access-rj2ww") pod "15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" (UID: "15860fa8-b5c0-4a1e-8c08-ff53afdd6e21"). InnerVolumeSpecName "kube-api-access-rj2ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:24:31.176821 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.176795    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" (UID: "15860fa8-b5c0-4a1e-8c08-ff53afdd6e21"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:24:31.267294 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.267260    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj2ww\" (UniqueName: \"kubernetes.io/projected/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-kube-api-access-rj2ww\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:24:31.267294 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.267288    2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21-tls-cert\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:24:31.337798 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.337762    2568 generic.go:356] "Generic (PLEG): container finished" podID="15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" containerID="3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52" exitCode=0
Jul 07 02:24:31.337798 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.337800    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-955598c66-jvhqt"
Jul 07 02:24:31.338003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.337847    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-955598c66-jvhqt" event={"ID":"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21","Type":"ContainerDied","Data":"3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52"}
Jul 07 02:24:31.338003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.337886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-955598c66-jvhqt" event={"ID":"15860fa8-b5c0-4a1e-8c08-ff53afdd6e21","Type":"ContainerDied","Data":"f3d79f511c54194df3ad47c2274a64cdef92e56868ef37fec00cb36d41b76698"}
Jul 07 02:24:31.338003 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.337907    2568 scope.go:117] "RemoveContainer" containerID="3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52"
Jul 07 02:24:31.347262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.347246    2568 scope.go:117] "RemoveContainer" containerID="3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52"
Jul 07 02:24:31.347516 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:24:31.347482    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52\": container with ID starting with 3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52 not found: ID does not exist" containerID="3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52"
Jul 07 02:24:31.347588 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.347528    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52"} err="failed to get container status \"3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52\": rpc error: code = NotFound desc = could not find container \"3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52\": container with ID starting with 3a2303257be60500e7eff2d60a2afaca02fb62593fa401008ef2285a806eff52 not found: ID does not exist"
Jul 07 02:24:31.369076 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.369048    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-955598c66-jvhqt"]
Jul 07 02:24:31.372101 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.372079    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-955598c66-jvhqt"]
Jul 07 02:24:31.426140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:31.426065    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:24:32.433142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:32.433109    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" path="/var/lib/kubelet/pods/15860fa8-b5c0-4a1e-8c08-ff53afdd6e21/volumes"
Jul 07 02:24:52.484230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:52.484191    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:24:52.489712 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:52.489689    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:24:55.635221 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:24:55.635191    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:25:07.468325 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:07.468285    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:25:11.662306 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:11.662262    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:25:15.247147 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:15.247111    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:25:19.918736 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:19.918700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:25:25.672707 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:25.672675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:25:25.783556 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:25.783519    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:25:25.866257 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:25.866228    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:25:37.542881 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:37.542846    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:25:43.788050 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:43.788013    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:25:45.072905 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:45.072866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:25:45.434304 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:25:45.434219    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:26:08.833808 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:08.833770    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:26:20.643684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:20.643605    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:26:25.668796 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:25.668748    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:25.768138 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:25.768088    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:25.861517 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:25.861460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:31.839702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:31.839666    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:31.944933 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:31.944894    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:32.041828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:32.041781    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:32.192384 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:32.192296    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:35.926828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:35.926790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:38.253375 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:38.253340    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:38.341199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:38.341162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:26:50.343252 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:50.343217    2568 scope.go:117] "RemoveContainer" containerID="b720e1c742b7118ccbce399991aca9cde2ae697c8979c6e876119d51b9e946ad"
Jul 07 02:26:55.325749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:26:55.325710    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:27:04.525455 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:27:04.525426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:27:12.928270 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:27:12.928230    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:27:21.722221 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:27:21.722185    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:27:32.218728 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:27:32.218693    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:27:40.826004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:27:40.825967    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:27:51.637910 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:27:51.637818    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:28:01.018848 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:28:01.018809    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:28:23.746538 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:28:23.746505    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:28:28.417043 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:28:28.416997    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:28:34.018318 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:28:34.018285    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:28:56.919756 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:28:56.919711    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:29:07.025189 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:07.025157    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:29:23.126006 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:23.125972    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:29:34.115849 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:34.115815    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:29:43.335074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:43.335040    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:29:52.331035 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:52.331000    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:29:52.514091 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:52.514066    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:29:52.521305 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:29:52.521284    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:30:01.327223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:01.327190    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:30:27.600258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:27.600223    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"]
Jul 07 02:30:27.600723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:27.600425    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m" podUID="e3d9707a-3eca-4cf7-b676-d1b2c366a8df" containerName="manager" containerID="cri-o://bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798" gracePeriod=10
Jul 07 02:30:29.856547 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:29.856519    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:30:29.944394 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:29.944311    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp4zm\" (UniqueName: \"kubernetes.io/projected/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-kube-api-access-zp4zm\") pod \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") "
Jul 07 02:30:29.944394 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:29.944375    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-extensions-socket-volume\") pod \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\" (UID: \"e3d9707a-3eca-4cf7-b676-d1b2c366a8df\") "
Jul 07 02:30:29.944792 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:29.944770    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e3d9707a-3eca-4cf7-b676-d1b2c366a8df" (UID: "e3d9707a-3eca-4cf7-b676-d1b2c366a8df"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:30:29.946404 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:29.946385    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-kube-api-access-zp4zm" (OuterVolumeSpecName: "kube-api-access-zp4zm") pod "e3d9707a-3eca-4cf7-b676-d1b2c366a8df" (UID: "e3d9707a-3eca-4cf7-b676-d1b2c366a8df"). InnerVolumeSpecName "kube-api-access-zp4zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:30:30.045298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.045264    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp4zm\" (UniqueName: \"kubernetes.io/projected/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-kube-api-access-zp4zm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:30:30.045298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.045290    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3d9707a-3eca-4cf7-b676-d1b2c366a8df-extensions-socket-volume\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:30:30.119749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.119716    2568 generic.go:356] "Generic (PLEG): container finished" podID="e3d9707a-3eca-4cf7-b676-d1b2c366a8df" containerID="bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798" exitCode=0
Jul 07 02:30:30.119909 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.119763    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"
Jul 07 02:30:30.119909 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.119796    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m" event={"ID":"e3d9707a-3eca-4cf7-b676-d1b2c366a8df","Type":"ContainerDied","Data":"bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798"}
Jul 07 02:30:30.119909 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.119848    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m" event={"ID":"e3d9707a-3eca-4cf7-b676-d1b2c366a8df","Type":"ContainerDied","Data":"9d4685ea4995e0c598030ba843fc532cc91fd671a114025faf44859785500109"}
Jul 07 02:30:30.119909 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.119870    2568 scope.go:117] "RemoveContainer" containerID="bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798"
Jul 07 02:30:30.130104 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.130089    2568 scope.go:117] "RemoveContainer" containerID="bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798"
Jul 07 02:30:30.130372 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:30:30.130355    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798\": container with ID starting with bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798 not found: ID does not exist" containerID="bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798"
Jul 07 02:30:30.130421 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.130380    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798"} err="failed to get container status \"bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798\": rpc error: code = NotFound desc = could not find container \"bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798\": container with ID starting with bc12f2f6f689a08fc245cc1e45ad5eb7f43f332faf2e268e7d50b10cb650d798 not found: ID does not exist"
Jul 07 02:30:30.141996 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.141971    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"]
Jul 07 02:30:30.147991 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.147969    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tzk4m"]
Jul 07 02:30:30.433595 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:30:30.433561    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d9707a-3eca-4cf7-b676-d1b2c366a8df" path="/var/lib/kubelet/pods/e3d9707a-3eca-4cf7-b676-d1b2c366a8df/volumes"
Jul 07 02:31:28.734434 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734395    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"]
Jul 07 02:31:28.734895 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734801    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" containerName="authorino"
Jul 07 02:31:28.734895 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734813    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" containerName="authorino"
Jul 07 02:31:28.734895 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734829    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d9707a-3eca-4cf7-b676-d1b2c366a8df" containerName="manager"
Jul 07 02:31:28.734895 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734834    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d9707a-3eca-4cf7-b676-d1b2c366a8df" containerName="manager"
Jul 07 02:31:28.735031 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734907    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="15860fa8-b5c0-4a1e-8c08-ff53afdd6e21" containerName="authorino"
Jul 07 02:31:28.735031 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.734916    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d9707a-3eca-4cf7-b676-d1b2c366a8df" containerName="manager"
Jul 07 02:31:28.737834 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.737818    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:28.741462 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.741444    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q7vvz\""
Jul 07 02:31:28.752273 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.752248    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"]
Jul 07 02:31:28.766926 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.766889    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c6fc69b-de8f-4447-8aa1-3834c84f4842-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2szcc\" (UID: \"2c6fc69b-de8f-4447-8aa1-3834c84f4842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:28.766926 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.766929    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxj8p\" (UniqueName: \"kubernetes.io/projected/2c6fc69b-de8f-4447-8aa1-3834c84f4842-kube-api-access-nxj8p\") pod \"kuadrant-operator-controller-manager-86878c769f-2szcc\" (UID: \"2c6fc69b-de8f-4447-8aa1-3834c84f4842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:28.868178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.868138    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c6fc69b-de8f-4447-8aa1-3834c84f4842-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2szcc\" (UID: \"2c6fc69b-de8f-4447-8aa1-3834c84f4842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:28.868178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.868183    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxj8p\" (UniqueName: \"kubernetes.io/projected/2c6fc69b-de8f-4447-8aa1-3834c84f4842-kube-api-access-nxj8p\") pod \"kuadrant-operator-controller-manager-86878c769f-2szcc\" (UID: \"2c6fc69b-de8f-4447-8aa1-3834c84f4842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:28.868624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.868603    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c6fc69b-de8f-4447-8aa1-3834c84f4842-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2szcc\" (UID: \"2c6fc69b-de8f-4447-8aa1-3834c84f4842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:28.879116 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:28.879086    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxj8p\" (UniqueName: \"kubernetes.io/projected/2c6fc69b-de8f-4447-8aa1-3834c84f4842-kube-api-access-nxj8p\") pod \"kuadrant-operator-controller-manager-86878c769f-2szcc\" (UID: \"2c6fc69b-de8f-4447-8aa1-3834c84f4842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:29.047864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:29.047825    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:29.203469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:29.203438    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"]
Jul 07 02:31:29.205671 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:31:29.205647    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6fc69b_de8f_4447_8aa1_3834c84f4842.slice/crio-aa380a3c7f1727b15ae5a6a404a7efd15d005a0d7031885dab755fcc392630a5 WatchSource:0}: Error finding container aa380a3c7f1727b15ae5a6a404a7efd15d005a0d7031885dab755fcc392630a5: Status 404 returned error can't find the container with id aa380a3c7f1727b15ae5a6a404a7efd15d005a0d7031885dab755fcc392630a5
Jul 07 02:31:29.400764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:29.400667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc" event={"ID":"2c6fc69b-de8f-4447-8aa1-3834c84f4842","Type":"ContainerStarted","Data":"7a6dcd360f030774ea3879122ccbd6bfe5f81bbb50e7c55c66ca942a9f53fc36"}
Jul 07 02:31:29.400764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:29.400722    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:29.400764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:29.400739    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc" event={"ID":"2c6fc69b-de8f-4447-8aa1-3834c84f4842","Type":"ContainerStarted","Data":"aa380a3c7f1727b15ae5a6a404a7efd15d005a0d7031885dab755fcc392630a5"}
Jul 07 02:31:29.423515 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:29.423449    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc" podStartSLOduration=1.423431166 podStartE2EDuration="1.423431166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:31:29.421519511 +0000 UTC m=+1299.624940249" watchObservedRunningTime="2026-07-07 02:31:29.423431166 +0000 UTC m=+1299.626851910"
Jul 07 02:31:40.407275 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:40.407241    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2szcc"
Jul 07 02:31:48.618119 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:48.618073    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:31:52.431554 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:52.431519    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:31:53.679131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:53.679077    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:31:59.826731 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:31:59.826694    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:32:08.019373 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:08.019333    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:32:19.231783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:19.231703    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:32:28.418141 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:28.418107    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:32:39.517238 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:39.517200    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:32:43.970946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:43.970910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:32:44.032241 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:44.032210    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:32:49.124755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:49.124721    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:32:59.522274 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:32:59.522239    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:33:11.226780 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:33:11.226740    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:33:20.724094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:33:20.724054    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:33:30.811929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:33:30.811896    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:33:43.936727 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:33:43.936650    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:33:52.433416 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:33:52.433378    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:34:00.823790 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:00.823754    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:34:11.722064 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:11.722029    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:34:22.623187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:22.623151    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:34:37.202315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:37.202278    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:34:39.226571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:39.226538    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:34:50.017216 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:50.017184    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:34:52.550239 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:52.550125    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:34:52.556771 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:34:52.556749    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:35:01.316169 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:35:01.316133    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:35:10.628878 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:35:10.628843    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:35:19.322234 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:35:19.322146    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:35:30.415543 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:35:30.415510    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:35:43.719331 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:35:43.719294    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:35:52.426109 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:35:52.426072    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:36:01.926565 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:36:01.926529    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:36:12.625601 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:36:12.625565    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:36:24.716036 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:36:24.715993    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:36:34.023332 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:36:34.023290    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:36:46.328448 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:36:46.328371    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:36:57.818070 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:36:57.818034    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:37:07.917583 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:07.917545    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:37:09.594591 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:09.594547    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:16.986215 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:16.986179    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:20.318047 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:20.318005    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:37:25.261326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:25.261289    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:28.825479 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:28.825443    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:37:30.369478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:30.369425    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:33.180252 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:33.180212    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:33.286611 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:33.286582    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:33.676689 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:33.676641    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:39.432383 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:39.432339    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:39.495568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:39.495534    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:39.636937 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:39.636902    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:39.699334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:39.699250    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:40.017859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:40.017825    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:37:40.832548 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:40.832513    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:40.925472 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:40.925435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:41.030685 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:41.030655    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:41.454710 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:41.454676    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:41.545717 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:41.545685    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:41.647663 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:41.647629    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:43.082670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:43.082640    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:45.930807 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:45.930773    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:46.153565 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:46.153534    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:46.246338 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:46.246310    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:46.373904 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:46.373866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:47.820854 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:47.820821    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:52.053257 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:52.053210    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:52.150158 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:52.150119    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:52.350835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:52.350753    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:52.462504 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:52.462470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:53.491779 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.491742    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:53.587974 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.587940    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"]
Jul 07 02:37:53.591418 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.591397    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.597334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.597304    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\""
Jul 07 02:37:53.597334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.597315    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-144e32df-serving-cert\""
Jul 07 02:37:53.599777 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.599750    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:53.620743 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.620707    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"]
Jul 07 02:37:53.631819 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.631763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhpk\" (UniqueName: \"kubernetes.io/projected/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-kube-api-access-cqhpk\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.631990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.631967    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.632031 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.631996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-openshift-service-ca\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.696802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.696770    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:53.732859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.732818    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.732859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.732863    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-openshift-service-ca\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.733096 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:37:53.732955    2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-144e32df-serving-cert: secret "maas-api-e2e-ait-144e32df-serving-cert" not found
Jul 07 02:37:53.733096 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:37:53.733011    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls podName:45580b03-4b89-4aaf-8f81-d6e48c87c2c6 nodeName:}" failed. No retries permitted until 2026-07-07 02:37:54.232994859 +0000 UTC m=+1684.436415581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls") pod "maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" (UID: "45580b03-4b89-4aaf-8f81-d6e48c87c2c6") : secret "maas-api-e2e-ait-144e32df-serving-cert" not found
Jul 07 02:37:53.733096 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.733033    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhpk\" (UniqueName: \"kubernetes.io/projected/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-kube-api-access-cqhpk\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.733635 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.733615    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-openshift-service-ca\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:53.746282 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:53.746209    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhpk\" (UniqueName: \"kubernetes.io/projected/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-kube-api-access-cqhpk\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:54.080811 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.080717    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:54.186786 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.186751    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:54.237444 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.237411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:54.239990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.239961    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls\") pod \"maas-api-e2e-ait-144e32df-769556bbb9-dnnc2\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") " pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:54.283279 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.283237    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:54.503187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.503153    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:54.509739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.509698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:54.604316 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.604285    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:54.640199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.640055    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"]
Jul 07 02:37:54.643470 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:37:54.643442    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45580b03_4b89_4aaf_8f81_d6e48c87c2c6.slice/crio-6be82bf181a97c3307af76abfc19da922921d7a50cd3af1dc5f2de202836cb97 WatchSource:0}: Error finding container 6be82bf181a97c3307af76abfc19da922921d7a50cd3af1dc5f2de202836cb97: Status 404 returned error can't find the container with id 6be82bf181a97c3307af76abfc19da922921d7a50cd3af1dc5f2de202836cb97
Jul 07 02:37:54.645198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:54.645179    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:37:55.277690 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:55.277619    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" event={"ID":"45580b03-4b89-4aaf-8f81-d6e48c87c2c6","Type":"ContainerStarted","Data":"6be82bf181a97c3307af76abfc19da922921d7a50cd3af1dc5f2de202836cb97"}
Jul 07 02:37:55.481180 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:55.481095    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:56.121449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:56.121420    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:37:56.285422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:56.285384    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" event={"ID":"45580b03-4b89-4aaf-8f81-d6e48c87c2c6","Type":"ContainerStarted","Data":"2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d"}
Jul 07 02:37:56.285634 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:56.285506    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:56.304200 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:56.304149    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" podStartSLOduration=1.8541476239999999 podStartE2EDuration="3.304131524s" totalImagesPullingTime="1.4499839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:37:56.303371067 +0000 UTC m=+1686.506791813" watchObservedRunningTime="2026-07-07 02:37:56.304131524 +0000 UTC m=+1686.507552299"
Jul 07 02:37:58.560639 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:58.560604    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:58.919118 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:58.919037    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"]
Jul 07 02:37:58.919303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:58.919216    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" podUID="45580b03-4b89-4aaf-8f81-d6e48c87c2c6" containerName="maas-api" containerID="cri-o://2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d" gracePeriod=30
Jul 07 02:37:58.928860 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:58.928809    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:37:59.174125 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.174061    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:59.286009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.285976    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-openshift-service-ca\") pod \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") "
Jul 07 02:37:59.286176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.286034    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls\") pod \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") "
Jul 07 02:37:59.286176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.286066    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqhpk\" (UniqueName: \"kubernetes.io/projected/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-kube-api-access-cqhpk\") pod \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\" (UID: \"45580b03-4b89-4aaf-8f81-d6e48c87c2c6\") "
Jul 07 02:37:59.286399 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.286379    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45580b03-4b89-4aaf-8f81-d6e48c87c2c6" (UID: "45580b03-4b89-4aaf-8f81-d6e48c87c2c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:37:59.288274 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.288251    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-kube-api-access-cqhpk" (OuterVolumeSpecName: "kube-api-access-cqhpk") pod "45580b03-4b89-4aaf-8f81-d6e48c87c2c6" (UID: "45580b03-4b89-4aaf-8f81-d6e48c87c2c6"). InnerVolumeSpecName "kube-api-access-cqhpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:37:59.288384 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.288336    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45580b03-4b89-4aaf-8f81-d6e48c87c2c6" (UID: "45580b03-4b89-4aaf-8f81-d6e48c87c2c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:37:59.304379 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.304320    2568 generic.go:356] "Generic (PLEG): container finished" podID="45580b03-4b89-4aaf-8f81-d6e48c87c2c6" containerID="2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d" exitCode=0
Jul 07 02:37:59.304559 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.304469    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"
Jul 07 02:37:59.304559 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.304483    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" event={"ID":"45580b03-4b89-4aaf-8f81-d6e48c87c2c6","Type":"ContainerDied","Data":"2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d"}
Jul 07 02:37:59.304669 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.304567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2" event={"ID":"45580b03-4b89-4aaf-8f81-d6e48c87c2c6","Type":"ContainerDied","Data":"6be82bf181a97c3307af76abfc19da922921d7a50cd3af1dc5f2de202836cb97"}
Jul 07 02:37:59.304669 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.304600    2568 scope.go:117] "RemoveContainer" containerID="2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d"
Jul 07 02:37:59.314683 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.314663    2568 scope.go:117] "RemoveContainer" containerID="2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d"
Jul 07 02:37:59.315038 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:37:59.315019    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d\": container with ID starting with 2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d not found: ID does not exist" containerID="2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d"
Jul 07 02:37:59.315085 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.315048    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d"} err="failed to get container status \"2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d\": rpc error: code = NotFound desc = could not find container \"2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d\": container with ID starting with 2e88b3ffacd078a8557da80e1628e5d38e1aad1177ea09b407d995847198c69d not found: ID does not exist"
Jul 07 02:37:59.327580 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.327545    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"]
Jul 07 02:37:59.333082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.333056    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-144e32df-769556bbb9-dnnc2"]
Jul 07 02:37:59.387759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.387715    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:37:59.387759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.387755    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqhpk\" (UniqueName: \"kubernetes.io/projected/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-kube-api-access-cqhpk\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:37:59.387759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:37:59.387768    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45580b03-4b89-4aaf-8f81-d6e48c87c2c6-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:38:00.433587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:00.433561    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45580b03-4b89-4aaf-8f81-d6e48c87c2c6" path="/var/lib/kubelet/pods/45580b03-4b89-4aaf-8f81-d6e48c87c2c6/volumes"
Jul 07 02:38:04.579280 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:04.579196    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:04.684192 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:04.684159    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:06.023395 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:06.023364    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:06.927713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:06.927678    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:38:10.529140 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:10.529109    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:10.628081 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:10.628051    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:10.748732 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:10.748698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:10.848682 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:10.848598    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:11.841845 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:11.841769    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:11.938672 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:11.938639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:12.036635 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:12.036600    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:12.242129 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:12.242104    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:12.635703 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:12.635606    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:12.736845 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:12.736801    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:14.049759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:14.049701    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:15.114548 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:15.114509    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:38:16.283348 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:16.283319    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:16.471612 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:16.471582    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:16.571725 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:16.571639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:16.672632 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:16.672600    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:18.035825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:18.035799    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:22.475911 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:22.475877    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:22.535846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:22.535815    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:22.933917 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:22.933882    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:38:22.967173 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:22.967146    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:23.065738 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:23.065706    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:23.199798 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:23.199722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:24.527285 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:24.527251    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:30.313384 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:30.313344    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:30.422918 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:30.422892    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:30.513119 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:30.513089    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:30.613220 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:30.613131    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:31.000728 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:31.000696    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:38:56.957716 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:38:56.957681    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:02.597756 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:02.597721    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:02.674767 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:02.674741    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:02.773988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:02.773953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:18.699520 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:18.699461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:24.755984 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:24.755953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:24.856995 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:24.856962    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:24.953864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:24.953816    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:35.486428 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:35.486391    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:36.286239 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:36.286204    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:42.195544 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:42.195457    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:42.265079 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:42.265049    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:42.353632 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:42.353600    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:42.450956 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:42.450869    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:42.553448 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:42.553405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:43.247981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:43.247941    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:43.345702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:43.345668    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:43.446296 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:43.446262    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:45.032785 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:45.032754    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:46.013221 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:46.013184    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:39:50.830471 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:50.830417    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:50.896148 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:50.896114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:51.498517 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:51.498464    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:52.581929 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:52.581814    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:39:52.590687 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:52.588733    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:39:55.125622 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:55.125581    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:39:57.302822 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:57.302767    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:57.403178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:57.403139    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:57.535639 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:57.535601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:57.860194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:57.860138    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:39:58.154194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:39:58.154163    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:01.092111 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.092077    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:01.190145 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.190114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:01.320235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.320202    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"]
Jul 07 02:40:01.320678 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.320658    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45580b03-4b89-4aaf-8f81-d6e48c87c2c6" containerName="maas-api"
Jul 07 02:40:01.320678 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.320678    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="45580b03-4b89-4aaf-8f81-d6e48c87c2c6" containerName="maas-api"
Jul 07 02:40:01.320882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.320770    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="45580b03-4b89-4aaf-8f81-d6e48c87c2c6" containerName="maas-api"
Jul 07 02:40:01.324132 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.324112    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.326742 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.326723    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-3a3e4d5a-serving-cert\""
Jul 07 02:40:01.326873 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.326741    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\""
Jul 07 02:40:01.336480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.336457    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"]
Jul 07 02:40:01.361114 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.361036    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:01.414422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.414386    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.414422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.414420    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl7rq\" (UniqueName: \"kubernetes.io/projected/d9b30281-f126-44db-918a-266e8b4ea2ea-kube-api-access-hl7rq\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.414701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.414464    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b30281-f126-44db-918a-266e8b4ea2ea-openshift-service-ca\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.515783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.515748    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.515783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.515784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl7rq\" (UniqueName: \"kubernetes.io/projected/d9b30281-f126-44db-918a-266e8b4ea2ea-kube-api-access-hl7rq\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.516046 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.515814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b30281-f126-44db-918a-266e8b4ea2ea-openshift-service-ca\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.516046 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:40:01.515934    2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-3a3e4d5a-serving-cert: secret "maas-api-e2e-mt-3a3e4d5a-serving-cert" not found
Jul 07 02:40:01.516046 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:40:01.516006    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls podName:d9b30281-f126-44db-918a-266e8b4ea2ea nodeName:}" failed. No retries permitted until 2026-07-07 02:40:02.015987259 +0000 UTC m=+1812.219407985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls") pod "maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" (UID: "d9b30281-f126-44db-918a-266e8b4ea2ea") : secret "maas-api-e2e-mt-3a3e4d5a-serving-cert" not found
Jul 07 02:40:01.516419 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.516401    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b30281-f126-44db-918a-266e8b4ea2ea-openshift-service-ca\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.525080 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.525058    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl7rq\" (UniqueName: \"kubernetes.io/projected/d9b30281-f126-44db-918a-266e8b4ea2ea-kube-api-access-hl7rq\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:01.666004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.665928    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:01.849946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.849891    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:01.950992 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:01.950901    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:02.020297 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:02.020264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:02.022987 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:02.022964    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls\") pod \"maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") " pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:02.235964 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:02.235880    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:02.581691 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:02.581665    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"]
Jul 07 02:40:02.583697 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:40:02.583670    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b30281_f126_44db_918a_266e8b4ea2ea.slice/crio-f3e516423444ab610b407c9567bfb1905fafe536ccff47f55a68903e6db5261f WatchSource:0}: Error finding container f3e516423444ab610b407c9567bfb1905fafe536ccff47f55a68903e6db5261f: Status 404 returned error can't find the container with id f3e516423444ab610b407c9567bfb1905fafe536ccff47f55a68903e6db5261f
Jul 07 02:40:02.912035 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:02.911945    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" event={"ID":"d9b30281-f126-44db-918a-266e8b4ea2ea","Type":"ContainerStarted","Data":"f3e516423444ab610b407c9567bfb1905fafe536ccff47f55a68903e6db5261f"}
Jul 07 02:40:03.918230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:03.918195    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" event={"ID":"d9b30281-f126-44db-918a-266e8b4ea2ea","Type":"ContainerStarted","Data":"ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7"}
Jul 07 02:40:03.918637 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:03.918301    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:03.937517 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:03.937459    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" podStartSLOduration=2.536469646 podStartE2EDuration="2.937445324s" totalImagesPullingTime="400.975678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:40:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:40:03.935983968 +0000 UTC m=+1814.139404713" watchObservedRunningTime="2026-07-07 02:40:03.937445324 +0000 UTC m=+1814.140866067"
Jul 07 02:40:05.482197 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:05.482153    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:05.822006 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:05.821971    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:40:06.172053 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:06.171947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:06.310166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:06.310106    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:06.577757 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:06.577697    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:09.928518 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:09.928477    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:12.330591 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:12.330553    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:12.429178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:12.429141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:12.725424 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:12.725337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:13.064923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.064885    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"]
Jul 07 02:40:13.065534 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.065461    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" podUID="d9b30281-f126-44db-918a-266e8b4ea2ea" containerName="maas-api" containerID="cri-o://ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7" gracePeriod=30
Jul 07 02:40:13.099876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.099834    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:13.195067 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.195032    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:13.313719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.313696    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:13.425024 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.424930    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls\") pod \"d9b30281-f126-44db-918a-266e8b4ea2ea\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") "
Jul 07 02:40:13.425024 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.424987    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl7rq\" (UniqueName: \"kubernetes.io/projected/d9b30281-f126-44db-918a-266e8b4ea2ea-kube-api-access-hl7rq\") pod \"d9b30281-f126-44db-918a-266e8b4ea2ea\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") "
Jul 07 02:40:13.425024 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.425010    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b30281-f126-44db-918a-266e8b4ea2ea-openshift-service-ca\") pod \"d9b30281-f126-44db-918a-266e8b4ea2ea\" (UID: \"d9b30281-f126-44db-918a-266e8b4ea2ea\") "
Jul 07 02:40:13.425460 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.425432    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b30281-f126-44db-918a-266e8b4ea2ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9b30281-f126-44db-918a-266e8b4ea2ea" (UID: "d9b30281-f126-44db-918a-266e8b4ea2ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:40:13.427144 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.427114    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9b30281-f126-44db-918a-266e8b4ea2ea" (UID: "d9b30281-f126-44db-918a-266e8b4ea2ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:40:13.427242 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.427123    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b30281-f126-44db-918a-266e8b4ea2ea-kube-api-access-hl7rq" (OuterVolumeSpecName: "kube-api-access-hl7rq") pod "d9b30281-f126-44db-918a-266e8b4ea2ea" (UID: "d9b30281-f126-44db-918a-266e8b4ea2ea"). InnerVolumeSpecName "kube-api-access-hl7rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:40:13.525959 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.525917    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9b30281-f126-44db-918a-266e8b4ea2ea-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:40:13.525959 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.525949    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl7rq\" (UniqueName: \"kubernetes.io/projected/d9b30281-f126-44db-918a-266e8b4ea2ea-kube-api-access-hl7rq\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:40:13.525959 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.525964    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9b30281-f126-44db-918a-266e8b4ea2ea-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:40:13.970313 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.970261    2568 generic.go:356] "Generic (PLEG): container finished" podID="d9b30281-f126-44db-918a-266e8b4ea2ea" containerID="ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7" exitCode=0
Jul 07 02:40:13.970512 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.970321    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"
Jul 07 02:40:13.970512 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.970338    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" event={"ID":"d9b30281-f126-44db-918a-266e8b4ea2ea","Type":"ContainerDied","Data":"ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7"}
Jul 07 02:40:13.970512 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.970391    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng" event={"ID":"d9b30281-f126-44db-918a-266e8b4ea2ea","Type":"ContainerDied","Data":"f3e516423444ab610b407c9567bfb1905fafe536ccff47f55a68903e6db5261f"}
Jul 07 02:40:13.970512 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.970420    2568 scope.go:117] "RemoveContainer" containerID="ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7"
Jul 07 02:40:13.980835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.980814    2568 scope.go:117] "RemoveContainer" containerID="ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7"
Jul 07 02:40:13.981196 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:40:13.981169    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7\": container with ID starting with ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7 not found: ID does not exist" containerID="ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7"
Jul 07 02:40:13.981303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:13.981207    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7"} err="failed to get container status \"ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7\": rpc error: code = NotFound desc = could not find container \"ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7\": container with ID starting with ad50264af748cef769983fde3160fb748ed3f1d3cfc24939fbaf629dab9d5af7 not found: ID does not exist"
Jul 07 02:40:14.005302 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:14.005256    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"]
Jul 07 02:40:14.007767 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:14.007743    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-3a3e4d5a-69d4cb8977-92sng"]
Jul 07 02:40:14.433170 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:14.433132    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9b30281-f126-44db-918a-266e8b4ea2ea" path="/var/lib/kubelet/pods/d9b30281-f126-44db-918a-266e8b4ea2ea/volumes"
Jul 07 02:40:16.919925 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:16.919887    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:40:19.038853 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.038820    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.138664 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.138631    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.313476 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.313381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.412463 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.412428    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.527597 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.527564    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.651952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.651853    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.749124 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.749085    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:19.858346 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:19.858314    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:25.574439 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:25.574394    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:25.677903 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:25.677869    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:25.774489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:25.774457    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:25.933265 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:25.933179    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:31.761844 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:31.761813    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:31.825161 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:31.825131    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:38.544392 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:38.544355    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:38.860406 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:38.860290    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:38.957783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:38.957739    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:42.040588 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.040550    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:42.113194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.113155    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"]
Jul 07 02:40:42.113647 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.113631    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9b30281-f126-44db-918a-266e8b4ea2ea" containerName="maas-api"
Jul 07 02:40:42.113737 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.113649    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9b30281-f126-44db-918a-266e8b4ea2ea" containerName="maas-api"
Jul 07 02:40:42.113737 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.113727    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9b30281-f126-44db-918a-266e8b4ea2ea" containerName="maas-api"
Jul 07 02:40:42.117138 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.117114    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.119519 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.119488    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\""
Jul 07 02:40:42.119675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.119653    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-717b0bf6-serving-cert\""
Jul 07 02:40:42.126967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.126941    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"]
Jul 07 02:40:42.128557 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.128525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:42.191614 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.191580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwgr\" (UniqueName: \"kubernetes.io/projected/53103e23-9365-4873-a7dc-e33ee0074ecd-kube-api-access-xlwgr\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.191769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.191626    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53103e23-9365-4873-a7dc-e33ee0074ecd-maas-api-tls\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.191769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.191713    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53103e23-9365-4873-a7dc-e33ee0074ecd-openshift-service-ca\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.193010 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.192987    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:42.292708 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.292615    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53103e23-9365-4873-a7dc-e33ee0074ecd-openshift-service-ca\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.292859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.292719    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwgr\" (UniqueName: \"kubernetes.io/projected/53103e23-9365-4873-a7dc-e33ee0074ecd-kube-api-access-xlwgr\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.292859 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.292763    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53103e23-9365-4873-a7dc-e33ee0074ecd-maas-api-tls\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.293330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.293302    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53103e23-9365-4873-a7dc-e33ee0074ecd-openshift-service-ca\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.294885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.294859    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:42.295226 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.295207    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53103e23-9365-4873-a7dc-e33ee0074ecd-maas-api-tls\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.302166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.302139    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwgr\" (UniqueName: \"kubernetes.io/projected/53103e23-9365-4873-a7dc-e33ee0074ecd-kube-api-access-xlwgr\") pod \"maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") " pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.429965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.429933    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:42.565410 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.565324    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"]
Jul 07 02:40:42.569486 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:40:42.569453    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53103e23_9365_4873_a7dc_e33ee0074ecd.slice/crio-00ad1e870b8cd9f04c64450c5345bc5147eafee7efe3fe8581aa24b25248f76e WatchSource:0}: Error finding container 00ad1e870b8cd9f04c64450c5345bc5147eafee7efe3fe8581aa24b25248f76e: Status 404 returned error can't find the container with id 00ad1e870b8cd9f04c64450c5345bc5147eafee7efe3fe8581aa24b25248f76e
Jul 07 02:40:42.603739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.603711    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:42.706770 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:42.706739    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:43.120634 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:43.120601    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4" event={"ID":"53103e23-9365-4873-a7dc-e33ee0074ecd","Type":"ContainerStarted","Data":"c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8"}
Jul 07 02:40:43.121022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:43.120653    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4" event={"ID":"53103e23-9365-4873-a7dc-e33ee0074ecd","Type":"ContainerStarted","Data":"00ad1e870b8cd9f04c64450c5345bc5147eafee7efe3fe8581aa24b25248f76e"}
Jul 07 02:40:43.121022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:43.120707    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:43.141533 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:43.141466    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4" podStartSLOduration=0.74109461 podStartE2EDuration="1.141451189s" totalImagesPullingTime="400.356579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:40:43.139687622 +0000 UTC m=+1853.343108377" watchObservedRunningTime="2026-07-07 02:40:43.141451189 +0000 UTC m=+1853.344871931"
Jul 07 02:40:46.535868 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:46.535829    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:48.323562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:48.323526    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:40:49.131662 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:49.131635    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:40:58.826992 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.826955    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"]
Jul 07 02:40:58.830924 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.830904    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:58.834739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.834718    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-717b0bf6\"/\"openshift-service-ca.crt\""
Jul 07 02:40:58.834860 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.834750    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-717b0bf6\"/\"e2e-lifecycle-model-717b0bf6-kserve-self-signed-certs\""
Jul 07 02:40:58.834860 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.834842    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-717b0bf6\"/\"kube-root-ca.crt\""
Jul 07 02:40:58.835009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.834993    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-717b0bf6\"/\"default-dockercfg-psrkd\""
Jul 07 02:40:58.840709 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.840689    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"]
Jul 07 02:40:58.867983 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.867948    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:40:58.948259 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.948226    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-model-cache\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:58.948435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.948288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-dshm\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:58.948435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.948330    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-home\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:58.948435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.948351    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tmp-dir\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:58.948435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.948366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tls-certs\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:58.948435 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:58.948418    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvk2l\" (UniqueName: \"kubernetes.io/projected/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-kube-api-access-dvk2l\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.049786 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.049744    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-model-cache\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.049937 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.049829    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-dshm\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.049937 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.049896    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-home\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.050030 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.049994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tmp-dir\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.050066 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.050028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tls-certs\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.050066 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.050053    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvk2l\" (UniqueName: \"kubernetes.io/projected/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-kube-api-access-dvk2l\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.050243 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.050222    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-model-cache\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.050243 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.050238    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-home\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.050373 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.050311    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tmp-dir\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.052221 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.052197    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-dshm\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.052434 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.052414    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tls-certs\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.059013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.058988    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvk2l\" (UniqueName: \"kubernetes.io/projected/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-kube-api-access-dvk2l\") pod \"e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") " pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.142446 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.142348    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:40:59.286201 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:40:59.286095    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"]
Jul 07 02:40:59.288503 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:40:59.288467    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ee5ddc2_2a08_4bea_b0fa_9abf07f513ce.slice/crio-ae518edd0a394aa812697c5e937e042924d81415ee20264c010d02aff2d573d2 WatchSource:0}: Error finding container ae518edd0a394aa812697c5e937e042924d81415ee20264c010d02aff2d573d2: Status 404 returned error can't find the container with id ae518edd0a394aa812697c5e937e042924d81415ee20264c010d02aff2d573d2
Jul 07 02:41:00.213181 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:00.213146    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp" event={"ID":"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce","Type":"ContainerStarted","Data":"555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38"}
Jul 07 02:41:00.213607 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:00.213190    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp" event={"ID":"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce","Type":"ContainerStarted","Data":"ae518edd0a394aa812697c5e937e042924d81415ee20264c010d02aff2d573d2"}
Jul 07 02:41:00.233405 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:00.233359    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp" podStartSLOduration=2.233343579 podStartE2EDuration="2.233343579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:41:00.232226374 +0000 UTC m=+1870.435647119" watchObservedRunningTime="2026-07-07 02:41:00.233343579 +0000 UTC m=+1870.436764323"
Jul 07 02:41:00.823576 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:00.823540    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:41:07.377683 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:07.377652    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:07.482438 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:07.482408    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:09.142670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:09.142628    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:41:09.143195 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:09.142686    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:41:09.155453 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:09.155420    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:41:09.267753 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:09.267719    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:41:11.375285 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:11.375254    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:11.475907 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:11.475871    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:18.423697 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:18.423666    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:41:24.051885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:24.051848    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:24.509706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:24.509674    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:24.609182 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:24.609138    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:24.745336 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:24.745298    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"]
Jul 07 02:41:24.745609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:24.745578    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4" podUID="53103e23-9365-4873-a7dc-e33ee0074ecd" containerName="maas-api" containerID="cri-o://c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8" gracePeriod=30
Jul 07 02:41:24.770635 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:24.770608    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:25.000731 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.000707    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:41:25.094518 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.094404    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53103e23-9365-4873-a7dc-e33ee0074ecd-maas-api-tls\") pod \"53103e23-9365-4873-a7dc-e33ee0074ecd\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") "
Jul 07 02:41:25.094972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.094532    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53103e23-9365-4873-a7dc-e33ee0074ecd-openshift-service-ca\") pod \"53103e23-9365-4873-a7dc-e33ee0074ecd\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") "
Jul 07 02:41:25.094972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.094574    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlwgr\" (UniqueName: \"kubernetes.io/projected/53103e23-9365-4873-a7dc-e33ee0074ecd-kube-api-access-xlwgr\") pod \"53103e23-9365-4873-a7dc-e33ee0074ecd\" (UID: \"53103e23-9365-4873-a7dc-e33ee0074ecd\") "
Jul 07 02:41:25.094972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.094870    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53103e23-9365-4873-a7dc-e33ee0074ecd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53103e23-9365-4873-a7dc-e33ee0074ecd" (UID: "53103e23-9365-4873-a7dc-e33ee0074ecd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:41:25.096535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.096515    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53103e23-9365-4873-a7dc-e33ee0074ecd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53103e23-9365-4873-a7dc-e33ee0074ecd" (UID: "53103e23-9365-4873-a7dc-e33ee0074ecd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:41:25.096700 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.096680    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53103e23-9365-4873-a7dc-e33ee0074ecd-kube-api-access-xlwgr" (OuterVolumeSpecName: "kube-api-access-xlwgr") pod "53103e23-9365-4873-a7dc-e33ee0074ecd" (UID: "53103e23-9365-4873-a7dc-e33ee0074ecd"). InnerVolumeSpecName "kube-api-access-xlwgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:41:25.195267 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.195235    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53103e23-9365-4873-a7dc-e33ee0074ecd-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:25.195267 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.195262    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlwgr\" (UniqueName: \"kubernetes.io/projected/53103e23-9365-4873-a7dc-e33ee0074ecd-kube-api-access-xlwgr\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:25.195267 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.195272    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53103e23-9365-4873-a7dc-e33ee0074ecd-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:25.336560 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.336523    2568 generic.go:356] "Generic (PLEG): container finished" podID="53103e23-9365-4873-a7dc-e33ee0074ecd" containerID="c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8" exitCode=0
Jul 07 02:41:25.336723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.336568    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"
Jul 07 02:41:25.336723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.336598    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4" event={"ID":"53103e23-9365-4873-a7dc-e33ee0074ecd","Type":"ContainerDied","Data":"c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8"}
Jul 07 02:41:25.336723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.336659    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4" event={"ID":"53103e23-9365-4873-a7dc-e33ee0074ecd","Type":"ContainerDied","Data":"00ad1e870b8cd9f04c64450c5345bc5147eafee7efe3fe8581aa24b25248f76e"}
Jul 07 02:41:25.336723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.336688    2568 scope.go:117] "RemoveContainer" containerID="c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8"
Jul 07 02:41:25.345751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.345736    2568 scope.go:117] "RemoveContainer" containerID="c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8"
Jul 07 02:41:25.346018 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:41:25.345995    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8\": container with ID starting with c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8 not found: ID does not exist" containerID="c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8"
Jul 07 02:41:25.346070 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.346027    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8"} err="failed to get container status \"c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8\": rpc error: code = NotFound desc = could not find container \"c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8\": container with ID starting with c325ea9558b30aa9126904b14e9a39be7c698d2c0bd7a97e3758f207fc1bc6b8 not found: ID does not exist"
Jul 07 02:41:25.357614 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.357589    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"]
Jul 07 02:41:25.365413 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:25.365385    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-717b0bf6-64db5dbf46-lk6g4"]
Jul 07 02:41:26.433657 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:26.433624    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53103e23-9365-4873-a7dc-e33ee0074ecd" path="/var/lib/kubelet/pods/53103e23-9365-4873-a7dc-e33ee0074ecd/volumes"
Jul 07 02:41:27.919854 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:27.919819    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:41:30.146804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:30.146769    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:35.126198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:35.126158    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"]
Jul 07 02:41:35.221364 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:35.221330    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:41:40.129529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:40.129480    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:41:50.146869 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.146803    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp" podUID="1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" containerName="main" containerID="cri-o://555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38" gracePeriod=45
Jul 07 02:41:50.392009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.391984    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:41:50.460175 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.460091    2568 generic.go:356] "Generic (PLEG): container finished" podID="1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" containerID="555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38" exitCode=0
Jul 07 02:41:50.460175 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.460130    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"
Jul 07 02:41:50.460363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.460180    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp" event={"ID":"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce","Type":"ContainerDied","Data":"555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38"}
Jul 07 02:41:50.460363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.460231    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp" event={"ID":"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce","Type":"ContainerDied","Data":"ae518edd0a394aa812697c5e937e042924d81415ee20264c010d02aff2d573d2"}
Jul 07 02:41:50.460363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.460259    2568 scope.go:117] "RemoveContainer" containerID="555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38"
Jul 07 02:41:50.468560 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.468545    2568 scope.go:117] "RemoveContainer" containerID="555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38"
Jul 07 02:41:50.468815 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:41:50.468798    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38\": container with ID starting with 555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38 not found: ID does not exist" containerID="555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38"
Jul 07 02:41:50.468870 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.468824    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38"} err="failed to get container status \"555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38\": rpc error: code = NotFound desc = could not find container \"555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38\": container with ID starting with 555d725ab907cd1d8bcde1a332284ef1674d69d5011c12f902cc793b99eebb38 not found: ID does not exist"
Jul 07 02:41:50.534081 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534052    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tmp-dir\") pod \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") "
Jul 07 02:41:50.534194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534134    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-dshm\") pod \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") "
Jul 07 02:41:50.534194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534164    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tls-certs\") pod \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") "
Jul 07 02:41:50.534194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534191    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvk2l\" (UniqueName: \"kubernetes.io/projected/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-kube-api-access-dvk2l\") pod \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") "
Jul 07 02:41:50.534369 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534222    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-model-cache\") pod \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") "
Jul 07 02:41:50.534369 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534263    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-home\") pod \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\" (UID: \"1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce\") "
Jul 07 02:41:50.534369 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534318    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" (UID: "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:41:50.534592 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534568    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-home" (OuterVolumeSpecName: "home") pod "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" (UID: "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:41:50.534670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534655    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:50.534724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534680    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:50.534724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.534653    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" (UID: "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:41:50.536374 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.536355    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-dshm" (OuterVolumeSpecName: "dshm") pod "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" (UID: "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:41:50.536545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.536444    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" (UID: "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:41:50.536545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.536534    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-kube-api-access-dvk2l" (OuterVolumeSpecName: "kube-api-access-dvk2l") pod "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" (UID: "1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce"). InnerVolumeSpecName "kube-api-access-dvk2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:41:50.635896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.635863    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:50.635896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.635894    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:50.636082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.635906    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:50.636082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.635914    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvk2l\" (UniqueName: \"kubernetes.io/projected/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce-kube-api-access-dvk2l\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:41:50.782032 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.782003    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"]
Jul 07 02:41:50.790592 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:50.790572    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-717b0bf6/e2e-lifecycle-model-717b0bf6-kserve-5cfb6b98b7-qrzvp"]
Jul 07 02:41:52.433507 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:41:52.433469    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" path="/var/lib/kubelet/pods/1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce/volumes"
Jul 07 02:42:12.195931 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:12.195901    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:13.336744 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:13.336708    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:14.532401 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:14.532366    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:42:18.204400 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:18.204369    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:18.267135 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:18.267098    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:18.502028 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:18.501997    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:18.725296 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:18.725263    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:19.779702 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:19.779669    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:19.878333 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:19.878291    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:19.990965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:19.990921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:20.753197 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:20.753160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:20.850256 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:20.850216    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:20.951167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:20.951134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:22.293145 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:22.293117    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:23.328458 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:23.328425    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:42:28.242213 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:28.242151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:28.312928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:28.312900    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:28.930430 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:28.930396    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:34.703393 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:34.703360    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:34.815573 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:34.815539    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:34.908440 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:34.908409    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:36.321897 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:36.321865    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:42:45.424714 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:45.424635    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:42:45.784281 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:42:45.784246    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:06.743853 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:06.743818    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:12.405506 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:12.405469    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:12.470193 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:12.470160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:12.752205 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:12.752178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:13.071723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:13.071639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:13.203134 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:13.203106    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:15.907175 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:15.907142    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:16.007802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.007767    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:16.134923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.134890    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"]
Jul 07 02:43:16.135505 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.135476    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" containerName="main"
Jul 07 02:43:16.135571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.135512    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" containerName="main"
Jul 07 02:43:16.135571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.135534    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53103e23-9365-4873-a7dc-e33ee0074ecd" containerName="maas-api"
Jul 07 02:43:16.135571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.135543    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="53103e23-9365-4873-a7dc-e33ee0074ecd" containerName="maas-api"
Jul 07 02:43:16.135695 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.135673    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="53103e23-9365-4873-a7dc-e33ee0074ecd" containerName="maas-api"
Jul 07 02:43:16.135695 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.135685    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ee5ddc2-2a08-4bea-b0fa-9abf07f513ce" containerName="main"
Jul 07 02:43:16.139227 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.139206    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.142480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.142451    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\""
Jul 07 02:43:16.142805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.142786    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-8f3d77-serving-cert\""
Jul 07 02:43:16.156605 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.156575    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"]
Jul 07 02:43:16.169764 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.169735    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:16.203835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.203799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbaacb-e003-4d68-ba18-8eacefda8f50-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.203981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.203839    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s26c\" (UniqueName: \"kubernetes.io/projected/eafbaacb-e003-4d68-ba18-8eacefda8f50-kube-api-access-5s26c\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.203981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.203932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbaacb-e003-4d68-ba18-8eacefda8f50-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.305298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.305260    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbaacb-e003-4d68-ba18-8eacefda8f50-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.305298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.305299    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s26c\" (UniqueName: \"kubernetes.io/projected/eafbaacb-e003-4d68-ba18-8eacefda8f50-kube-api-access-5s26c\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.305558 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.305345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbaacb-e003-4d68-ba18-8eacefda8f50-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.305988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.305969    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbaacb-e003-4d68-ba18-8eacefda8f50-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.307675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.307650    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbaacb-e003-4d68-ba18-8eacefda8f50-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.316901 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.316875    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s26c\" (UniqueName: \"kubernetes.io/projected/eafbaacb-e003-4d68-ba18-8eacefda8f50-kube-api-access-5s26c\") pod \"maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") " pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.375716 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.375679    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:16.462738 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.462640    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:16.596138 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.596105    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"]
Jul 07 02:43:16.599839 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:43:16.599815    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeafbaacb_e003_4d68_ba18_8eacefda8f50.slice/crio-4772013db2d547d56c8b1d93d0dda05e59083bc772fe79ba398e045e075ba089 WatchSource:0}: Error finding container 4772013db2d547d56c8b1d93d0dda05e59083bc772fe79ba398e045e075ba089: Status 404 returned error can't find the container with id 4772013db2d547d56c8b1d93d0dda05e59083bc772fe79ba398e045e075ba089
Jul 07 02:43:16.601707 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.601686    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:43:16.729436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.729359    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:16.828082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.828053    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:16.882724 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:16.882689    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8" event={"ID":"eafbaacb-e003-4d68-ba18-8eacefda8f50","Type":"ContainerStarted","Data":"4772013db2d547d56c8b1d93d0dda05e59083bc772fe79ba398e045e075ba089"}
Jul 07 02:43:17.888872 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:17.888837    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8" event={"ID":"eafbaacb-e003-4d68-ba18-8eacefda8f50","Type":"ContainerStarted","Data":"d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e"}
Jul 07 02:43:17.889304 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:17.888979    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:17.910695 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:17.910642    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8" podStartSLOduration=1.436335491 podStartE2EDuration="1.910626691s" totalImagesPullingTime="474.2912ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:43:17.908087651 +0000 UTC m=+2008.111508395" watchObservedRunningTime="2026-07-07 02:43:17.910626691 +0000 UTC m=+2008.114047435"
Jul 07 02:43:20.406024 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:20.405995    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:20.730659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:20.730558    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:21.036456 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:21.036428    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:21.137081 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:21.137053    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:23.625721 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:23.625684    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:43:23.899680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:23.899604    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:43:23.980200 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:23.980157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:24.077611 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:24.077581    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:24.266674 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:24.266638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:24.366634 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:24.366591    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:24.743542 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:24.743512    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:24.825217 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:24.825184    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:28.453922 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:28.453869    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:33.917803 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:33.917771    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:43:34.322957 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.322926    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"]
Jul 07 02:43:34.326613 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.326595    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.328995 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.328969    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8f3d77\"/\"kube-root-ca.crt\""
Jul 07 02:43:34.330159 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.330139    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8f3d77\"/\"openshift-service-ca.crt\""
Jul 07 02:43:34.330271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.330148    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8f3d77\"/\"default-dockercfg-jkf2g\""
Jul 07 02:43:34.330339 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.330324    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8f3d77\"/\"test-model-8f3d77-kserve-self-signed-certs\""
Jul 07 02:43:34.336263 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.336243    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"]
Jul 07 02:43:34.370715 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.370688    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:34.379809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.379776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-home\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.379946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.379838    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-tmp-dir\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.379946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.379856    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q76c5\" (UniqueName: \"kubernetes.io/projected/3baa5f95-65ad-45e4-9c1c-c91c40370164-kube-api-access-q76c5\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.379946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.379908    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-dshm\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.380089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.379949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-model-cache\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.380089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.379967    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3baa5f95-65ad-45e4-9c1c-c91c40370164-tls-certs\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.480721 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.480680    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-home\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.480913 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.480816    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-tmp-dir\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.480913 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.480835    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q76c5\" (UniqueName: \"kubernetes.io/projected/3baa5f95-65ad-45e4-9c1c-c91c40370164-kube-api-access-q76c5\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.480913 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.480868    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-dshm\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.481094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.480926    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-model-cache\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.481094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.480953    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3baa5f95-65ad-45e4-9c1c-c91c40370164-tls-certs\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.481094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.481054    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-home\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.481244 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.481188    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-tmp-dir\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.481285 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.481249    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-model-cache\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.483155 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.483135    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-dshm\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.483326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.483309    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3baa5f95-65ad-45e4-9c1c-c91c40370164-tls-certs\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.489817 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.489795    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q76c5\" (UniqueName: \"kubernetes.io/projected/3baa5f95-65ad-45e4-9c1c-c91c40370164-kube-api-access-q76c5\") pod \"test-model-8f3d77-kserve-5848dbcdf5-2rf8z\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") " pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.637025 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.636944    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:34.775354 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.775328    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"]
Jul 07 02:43:34.777278 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:43:34.777241    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3baa5f95_65ad_45e4_9c1c_c91c40370164.slice/crio-41ff4abbaac3c07f10606891f242ddb7a2751f70a0d78feda92f3e61c0c15949 WatchSource:0}: Error finding container 41ff4abbaac3c07f10606891f242ddb7a2751f70a0d78feda92f3e61c0c15949: Status 404 returned error can't find the container with id 41ff4abbaac3c07f10606891f242ddb7a2751f70a0d78feda92f3e61c0c15949
Jul 07 02:43:34.974369 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.974270    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z" event={"ID":"3baa5f95-65ad-45e4-9c1c-c91c40370164","Type":"ContainerStarted","Data":"fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527"}
Jul 07 02:43:34.974369 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.974321    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z" event={"ID":"3baa5f95-65ad-45e4-9c1c-c91c40370164","Type":"ContainerStarted","Data":"41ff4abbaac3c07f10606891f242ddb7a2751f70a0d78feda92f3e61c0c15949"}
Jul 07 02:43:34.994450 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:34.994397    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z" podStartSLOduration=0.994382334 podStartE2EDuration="994.382334ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:43:34.992313169 +0000 UTC m=+2025.195733915" watchObservedRunningTime="2026-07-07 02:43:34.994382334 +0000 UTC m=+2025.197803086"
Jul 07 02:43:38.152233 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:38.152198    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:38.256172 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:38.256141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:43.015735 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:43.015700    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:43:44.637846 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:44.637810    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:44.638259 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:44.637861    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:44.650321 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:44.650294    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:45.034923 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:45.034895    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:43:45.521866 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:45.521834    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:45.623686 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:45.623654    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:57.326712 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.326678    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"]
Jul 07 02:43:57.330222 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.330201    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.333013 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.332989    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-902459\"/\"default-dockercfg-gkbtz\""
Jul 07 02:43:57.333138 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.333021    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-902459\"/\"kube-root-ca.crt\""
Jul 07 02:43:57.334106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.334088    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-902459\"/\"openshift-service-ca.crt\""
Jul 07 02:43:57.334228 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.334124    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-902459\"/\"test-model-902459-kserve-self-signed-certs\""
Jul 07 02:43:57.345036 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.345012    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"]
Jul 07 02:43:57.360205 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.360160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:43:57.391922 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.391897    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tls-certs\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.392078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.391929    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq8l4\" (UniqueName: \"kubernetes.io/projected/ee23f5f8-9512-4f1b-9aa9-7440089a449a-kube-api-access-qq8l4\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.392078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.391953    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-model-cache\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.392078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.392003    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-dshm\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.392187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.392112    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tmp-dir\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.392187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.392150    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-home\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493053 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493016    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-model-cache\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-dshm\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tmp-dir\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-home\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tls-certs\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493230 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq8l4\" (UniqueName: \"kubernetes.io/projected/ee23f5f8-9512-4f1b-9aa9-7440089a449a-kube-api-access-qq8l4\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493458 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493431    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-model-cache\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493570 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493545    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tmp-dir\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.493699 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.493622    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-home\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.495414 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.495392    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-dshm\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.495618 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.495603    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tls-certs\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.501164 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.501143    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq8l4\" (UniqueName: \"kubernetes.io/projected/ee23f5f8-9512-4f1b-9aa9-7440089a449a-kube-api-access-qq8l4\") pod \"test-model-902459-kserve-7d8d4c444c-kfpx6\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") " pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.647542 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.647432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:43:57.782135 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:57.782109    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"]
Jul 07 02:43:57.784337 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:43:57.784309    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee23f5f8_9512_4f1b_9aa9_7440089a449a.slice/crio-297d5a077920f463144c593e42d3a90136d2ff2d6006282851b70c30611a8c92 WatchSource:0}: Error finding container 297d5a077920f463144c593e42d3a90136d2ff2d6006282851b70c30611a8c92: Status 404 returned error can't find the container with id 297d5a077920f463144c593e42d3a90136d2ff2d6006282851b70c30611a8c92
Jul 07 02:43:58.094882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:58.094842    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6" event={"ID":"ee23f5f8-9512-4f1b-9aa9-7440089a449a","Type":"ContainerStarted","Data":"a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850"}
Jul 07 02:43:58.094882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:58.094887    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6" event={"ID":"ee23f5f8-9512-4f1b-9aa9-7440089a449a","Type":"ContainerStarted","Data":"297d5a077920f463144c593e42d3a90136d2ff2d6006282851b70c30611a8c92"}
Jul 07 02:43:58.112615 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:43:58.112569    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6" podStartSLOduration=1.112553268 podStartE2EDuration="1.112553268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:43:58.111647169 +0000 UTC m=+2048.315067927" watchObservedRunningTime="2026-07-07 02:43:58.112553268 +0000 UTC m=+2048.315974011"
Jul 07 02:44:01.416725 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:01.416680    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:44:07.647999 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:07.647951    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:44:07.648410 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:07.648039    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:44:07.660565 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:07.660539    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:44:08.155096 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:08.155069    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:44:17.518448 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:17.518353    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:44:20.293391 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:20.293351    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:20.394162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:20.394126    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:26.784027 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:26.783990    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"]
Jul 07 02:44:26.802684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:26.802650    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:27.641953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:27.641920    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"]
Jul 07 02:44:27.663015 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:27.662985    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:27.834699 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:27.834664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:28.213584 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.213545    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:28.316885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.316855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:28.417652 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.417621    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:28.510295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.510263    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"]
Jul 07 02:44:28.510488 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.510469    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8" podUID="eafbaacb-e003-4d68-ba18-8eacefda8f50" containerName="maas-api" containerID="cri-o://d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e" gracePeriod=30
Jul 07 02:44:28.557610 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.557574    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:28.765736 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.765662    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:44:28.893361 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.893332    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbaacb-e003-4d68-ba18-8eacefda8f50-maas-api-tls\") pod \"eafbaacb-e003-4d68-ba18-8eacefda8f50\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") "
Jul 07 02:44:28.893876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.893387    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbaacb-e003-4d68-ba18-8eacefda8f50-openshift-service-ca\") pod \"eafbaacb-e003-4d68-ba18-8eacefda8f50\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") "
Jul 07 02:44:28.893876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.893537    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s26c\" (UniqueName: \"kubernetes.io/projected/eafbaacb-e003-4d68-ba18-8eacefda8f50-kube-api-access-5s26c\") pod \"eafbaacb-e003-4d68-ba18-8eacefda8f50\" (UID: \"eafbaacb-e003-4d68-ba18-8eacefda8f50\") "
Jul 07 02:44:28.893876 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.893866    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eafbaacb-e003-4d68-ba18-8eacefda8f50-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eafbaacb-e003-4d68-ba18-8eacefda8f50" (UID: "eafbaacb-e003-4d68-ba18-8eacefda8f50"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:44:28.895459 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.895437    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eafbaacb-e003-4d68-ba18-8eacefda8f50-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eafbaacb-e003-4d68-ba18-8eacefda8f50" (UID: "eafbaacb-e003-4d68-ba18-8eacefda8f50"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:44:28.895573 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.895547    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafbaacb-e003-4d68-ba18-8eacefda8f50-kube-api-access-5s26c" (OuterVolumeSpecName: "kube-api-access-5s26c") pod "eafbaacb-e003-4d68-ba18-8eacefda8f50" (UID: "eafbaacb-e003-4d68-ba18-8eacefda8f50"). InnerVolumeSpecName "kube-api-access-5s26c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:44:28.995022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.994994    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s26c\" (UniqueName: \"kubernetes.io/projected/eafbaacb-e003-4d68-ba18-8eacefda8f50-kube-api-access-5s26c\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:28.995022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.995020    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbaacb-e003-4d68-ba18-8eacefda8f50-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:28.995223 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:28.995032    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbaacb-e003-4d68-ba18-8eacefda8f50-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:29.259770 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.259734    2568 generic.go:356] "Generic (PLEG): container finished" podID="eafbaacb-e003-4d68-ba18-8eacefda8f50" containerID="d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e" exitCode=0
Jul 07 02:44:29.259928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.259797    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"
Jul 07 02:44:29.259928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.259821    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8" event={"ID":"eafbaacb-e003-4d68-ba18-8eacefda8f50","Type":"ContainerDied","Data":"d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e"}
Jul 07 02:44:29.259928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.259904    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8" event={"ID":"eafbaacb-e003-4d68-ba18-8eacefda8f50","Type":"ContainerDied","Data":"4772013db2d547d56c8b1d93d0dda05e59083bc772fe79ba398e045e075ba089"}
Jul 07 02:44:29.260038 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.259933    2568 scope.go:117] "RemoveContainer" containerID="d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e"
Jul 07 02:44:29.269650 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.269630    2568 scope.go:117] "RemoveContainer" containerID="d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e"
Jul 07 02:44:29.269897 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:44:29.269879    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e\": container with ID starting with d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e not found: ID does not exist" containerID="d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e"
Jul 07 02:44:29.269965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.269910    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e"} err="failed to get container status \"d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e\": rpc error: code = NotFound desc = could not find container \"d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e\": container with ID starting with d49cae2651d093f1dc7a37a241fb6e2e9129c957c3b31a6038476fd335469b9e not found: ID does not exist"
Jul 07 02:44:29.283322 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.283296    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"]
Jul 07 02:44:29.286852 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:29.286833    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f3d77-c996445b4-bpbp8"]
Jul 07 02:44:30.221042 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:30.221009    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:44:30.435418 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:30.435390    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eafbaacb-e003-4d68-ba18-8eacefda8f50" path="/var/lib/kubelet/pods/eafbaacb-e003-4d68-ba18-8eacefda8f50/volumes"
Jul 07 02:44:33.283936 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:33.283880    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:44:41.805012 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:41.804963    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z" podUID="3baa5f95-65ad-45e4-9c1c-c91c40370164" containerName="main" containerID="cri-o://fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527" gracePeriod=45
Jul 07 02:44:42.056540 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.056463    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:44:42.123233 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123200    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-dshm\") pod \"3baa5f95-65ad-45e4-9c1c-c91c40370164\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") "
Jul 07 02:44:42.123233 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123236    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3baa5f95-65ad-45e4-9c1c-c91c40370164-tls-certs\") pod \"3baa5f95-65ad-45e4-9c1c-c91c40370164\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") "
Jul 07 02:44:42.123487 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123280    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-tmp-dir\") pod \"3baa5f95-65ad-45e4-9c1c-c91c40370164\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") "
Jul 07 02:44:42.123487 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123354    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-model-cache\") pod \"3baa5f95-65ad-45e4-9c1c-c91c40370164\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") "
Jul 07 02:44:42.123487 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123395    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q76c5\" (UniqueName: \"kubernetes.io/projected/3baa5f95-65ad-45e4-9c1c-c91c40370164-kube-api-access-q76c5\") pod \"3baa5f95-65ad-45e4-9c1c-c91c40370164\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") "
Jul 07 02:44:42.123487 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123416    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-home\") pod \"3baa5f95-65ad-45e4-9c1c-c91c40370164\" (UID: \"3baa5f95-65ad-45e4-9c1c-c91c40370164\") "
Jul 07 02:44:42.123738 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123681    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3baa5f95-65ad-45e4-9c1c-c91c40370164" (UID: "3baa5f95-65ad-45e4-9c1c-c91c40370164"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:42.123738 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123692    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-model-cache" (OuterVolumeSpecName: "model-cache") pod "3baa5f95-65ad-45e4-9c1c-c91c40370164" (UID: "3baa5f95-65ad-45e4-9c1c-c91c40370164"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:42.123870 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.123780    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-home" (OuterVolumeSpecName: "home") pod "3baa5f95-65ad-45e4-9c1c-c91c40370164" (UID: "3baa5f95-65ad-45e4-9c1c-c91c40370164"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:42.125568 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.125547    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-dshm" (OuterVolumeSpecName: "dshm") pod "3baa5f95-65ad-45e4-9c1c-c91c40370164" (UID: "3baa5f95-65ad-45e4-9c1c-c91c40370164"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:42.125676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.125580    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3baa5f95-65ad-45e4-9c1c-c91c40370164-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3baa5f95-65ad-45e4-9c1c-c91c40370164" (UID: "3baa5f95-65ad-45e4-9c1c-c91c40370164"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:44:42.125676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.125661    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3baa5f95-65ad-45e4-9c1c-c91c40370164-kube-api-access-q76c5" (OuterVolumeSpecName: "kube-api-access-q76c5") pod "3baa5f95-65ad-45e4-9c1c-c91c40370164" (UID: "3baa5f95-65ad-45e4-9c1c-c91c40370164"). InnerVolumeSpecName "kube-api-access-q76c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:44:42.224551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.224487    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:42.224551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.224547    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q76c5\" (UniqueName: \"kubernetes.io/projected/3baa5f95-65ad-45e4-9c1c-c91c40370164-kube-api-access-q76c5\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:42.224765 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.224563    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:42.224765 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.224576    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:42.224765 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.224586    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3baa5f95-65ad-45e4-9c1c-c91c40370164-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:42.224765 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.224598    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3baa5f95-65ad-45e4-9c1c-c91c40370164-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:42.326240 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.326155    2568 generic.go:356] "Generic (PLEG): container finished" podID="3baa5f95-65ad-45e4-9c1c-c91c40370164" containerID="fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527" exitCode=0
Jul 07 02:44:42.326240 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.326199    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"
Jul 07 02:44:42.326449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.326235    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z" event={"ID":"3baa5f95-65ad-45e4-9c1c-c91c40370164","Type":"ContainerDied","Data":"fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527"}
Jul 07 02:44:42.326449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.326278    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z" event={"ID":"3baa5f95-65ad-45e4-9c1c-c91c40370164","Type":"ContainerDied","Data":"41ff4abbaac3c07f10606891f242ddb7a2751f70a0d78feda92f3e61c0c15949"}
Jul 07 02:44:42.326449 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.326296    2568 scope.go:117] "RemoveContainer" containerID="fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527"
Jul 07 02:44:42.334899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.334885    2568 scope.go:117] "RemoveContainer" containerID="fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527"
Jul 07 02:44:42.335155 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:44:42.335132    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527\": container with ID starting with fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527 not found: ID does not exist" containerID="fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527"
Jul 07 02:44:42.335206 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.335167    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527"} err="failed to get container status \"fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527\": rpc error: code = NotFound desc = could not find container \"fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527\": container with ID starting with fea2666795f952b6035b9a1720538689fb0bb71003bfb2515fb5053a7e0ea527 not found: ID does not exist"
Jul 07 02:44:42.348074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.348052    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"]
Jul 07 02:44:42.353763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.353744    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8f3d77/test-model-8f3d77-kserve-5848dbcdf5-2rf8z"]
Jul 07 02:44:42.441060 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.441026    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3baa5f95-65ad-45e4-9c1c-c91c40370164" path="/var/lib/kubelet/pods/3baa5f95-65ad-45e4-9c1c-c91c40370164/volumes"
Jul 07 02:44:42.514258 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.514229    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:44:42.663682 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.663582    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6" podUID="ee23f5f8-9512-4f1b-9aa9-7440089a449a" containerName="main" containerID="cri-o://a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850" gracePeriod=45
Jul 07 02:44:42.922654 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:42.922595    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:44:43.031651 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031617    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-model-cache\") pod \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") "
Jul 07 02:44:43.031835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031661    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq8l4\" (UniqueName: \"kubernetes.io/projected/ee23f5f8-9512-4f1b-9aa9-7440089a449a-kube-api-access-qq8l4\") pod \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") "
Jul 07 02:44:43.031835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031707    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-dshm\") pod \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") "
Jul 07 02:44:43.031835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031730    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-home\") pod \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") "
Jul 07 02:44:43.031835 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031816    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tmp-dir\") pod \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") "
Jul 07 02:44:43.032062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031868    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tls-certs\") pod \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\" (UID: \"ee23f5f8-9512-4f1b-9aa9-7440089a449a\") "
Jul 07 02:44:43.032062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.031945    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee23f5f8-9512-4f1b-9aa9-7440089a449a" (UID: "ee23f5f8-9512-4f1b-9aa9-7440089a449a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:43.032062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.032043    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee23f5f8-9512-4f1b-9aa9-7440089a449a" (UID: "ee23f5f8-9512-4f1b-9aa9-7440089a449a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:43.032201 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.032064    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-home" (OuterVolumeSpecName: "home") pod "ee23f5f8-9512-4f1b-9aa9-7440089a449a" (UID: "ee23f5f8-9512-4f1b-9aa9-7440089a449a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:43.032242 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.032204    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:43.032242 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.032222    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:43.032242 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.032231    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:43.034128 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.034097    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee23f5f8-9512-4f1b-9aa9-7440089a449a-kube-api-access-qq8l4" (OuterVolumeSpecName: "kube-api-access-qq8l4") pod "ee23f5f8-9512-4f1b-9aa9-7440089a449a" (UID: "ee23f5f8-9512-4f1b-9aa9-7440089a449a"). InnerVolumeSpecName "kube-api-access-qq8l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:44:43.034262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.034128    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-dshm" (OuterVolumeSpecName: "dshm") pod "ee23f5f8-9512-4f1b-9aa9-7440089a449a" (UID: "ee23f5f8-9512-4f1b-9aa9-7440089a449a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:44:43.034262 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.034137    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee23f5f8-9512-4f1b-9aa9-7440089a449a" (UID: "ee23f5f8-9512-4f1b-9aa9-7440089a449a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:44:43.132802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.132770    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee23f5f8-9512-4f1b-9aa9-7440089a449a-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:43.132802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.132794    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq8l4\" (UniqueName: \"kubernetes.io/projected/ee23f5f8-9512-4f1b-9aa9-7440089a449a-kube-api-access-qq8l4\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:43.132802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.132804    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee23f5f8-9512-4f1b-9aa9-7440089a449a-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:44:43.333099 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.333064    2568 generic.go:356] "Generic (PLEG): container finished" podID="ee23f5f8-9512-4f1b-9aa9-7440089a449a" containerID="a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850" exitCode=0
Jul 07 02:44:43.333315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.333103    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"
Jul 07 02:44:43.333315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.333155    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6" event={"ID":"ee23f5f8-9512-4f1b-9aa9-7440089a449a","Type":"ContainerDied","Data":"a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850"}
Jul 07 02:44:43.333315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.333202    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6" event={"ID":"ee23f5f8-9512-4f1b-9aa9-7440089a449a","Type":"ContainerDied","Data":"297d5a077920f463144c593e42d3a90136d2ff2d6006282851b70c30611a8c92"}
Jul 07 02:44:43.333315 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.333225    2568 scope.go:117] "RemoveContainer" containerID="a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850"
Jul 07 02:44:43.342104 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.342085    2568 scope.go:117] "RemoveContainer" containerID="a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850"
Jul 07 02:44:43.342373 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:44:43.342357    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850\": container with ID starting with a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850 not found: ID does not exist" containerID="a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850"
Jul 07 02:44:43.342415 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.342382    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850"} err="failed to get container status \"a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850\": rpc error: code = NotFound desc = could not find container \"a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850\": container with ID starting with a3269c3d77cebb37dde84ea5586b78103df66df11ca97ecdc07e182364dd4850 not found: ID does not exist"
Jul 07 02:44:43.354389 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.354363    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"]
Jul 07 02:44:43.360842 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:43.360820    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-902459/test-model-902459-kserve-7d8d4c444c-kfpx6"]
Jul 07 02:44:44.432789 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:44.432755    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee23f5f8-9512-4f1b-9aa9-7440089a449a" path="/var/lib/kubelet/pods/ee23f5f8-9512-4f1b-9aa9-7440089a449a/volumes"
Jul 07 02:44:52.611123 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:52.611002    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:44:52.636684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:44:52.619392    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:45:10.009937 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:10.009900    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:10.303868 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:10.303753    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:10.527576 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:10.527533    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:10.700001 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:10.699902    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:11.090990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:11.090939    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:11.227691 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:11.227662    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:11.328365 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:11.328332    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:11.595739 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:11.595703    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:13.929719 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:13.929685    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:45:16.992381 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:16.992356    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:17.062584 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:17.062540    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:17.269333 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:17.269197    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:17.479706 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:17.479676    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:17.833513 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:17.833458    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:18.135814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:18.135726    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:18.533135 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:18.533100    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:20.718746 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.718710    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:20.786368 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786332    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"]
Jul 07 02:45:20.786806 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786788    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eafbaacb-e003-4d68-ba18-8eacefda8f50" containerName="maas-api"
Jul 07 02:45:20.786899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786809    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eafbaacb-e003-4d68-ba18-8eacefda8f50" containerName="maas-api"
Jul 07 02:45:20.786899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786825    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3baa5f95-65ad-45e4-9c1c-c91c40370164" containerName="main"
Jul 07 02:45:20.786899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786833    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3baa5f95-65ad-45e4-9c1c-c91c40370164" containerName="main"
Jul 07 02:45:20.786899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786862    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee23f5f8-9512-4f1b-9aa9-7440089a449a" containerName="main"
Jul 07 02:45:20.786899 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786872    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee23f5f8-9512-4f1b-9aa9-7440089a449a" containerName="main"
Jul 07 02:45:20.787141 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786978    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3baa5f95-65ad-45e4-9c1c-c91c40370164" containerName="main"
Jul 07 02:45:20.787141 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.786992    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="eafbaacb-e003-4d68-ba18-8eacefda8f50" containerName="maas-api"
Jul 07 02:45:20.787141 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.787001    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee23f5f8-9512-4f1b-9aa9-7440089a449a" containerName="main"
Jul 07 02:45:20.790203 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.790176    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.792769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.792736    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-8ad891-serving-cert\""
Jul 07 02:45:20.793005 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.792986    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\""
Jul 07 02:45:20.793243 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.793228    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:20.803873 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.803849    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"]
Jul 07 02:45:20.867819 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.867775    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:20.891984 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.891954    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2836af7a-f094-4a6f-bbb7-62ae5d544374-maas-api-tls\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.892126 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.892005    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plljq\" (UniqueName: \"kubernetes.io/projected/2836af7a-f094-4a6f-bbb7-62ae5d544374-kube-api-access-plljq\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.892126 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.892090    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2836af7a-f094-4a6f-bbb7-62ae5d544374-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.968070 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.968037    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:20.993769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.993739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2836af7a-f094-4a6f-bbb7-62ae5d544374-maas-api-tls\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.993907 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.993806    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plljq\" (UniqueName: \"kubernetes.io/projected/2836af7a-f094-4a6f-bbb7-62ae5d544374-kube-api-access-plljq\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.993907 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.993873    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2836af7a-f094-4a6f-bbb7-62ae5d544374-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.994590 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.994567    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2836af7a-f094-4a6f-bbb7-62ae5d544374-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:20.996128 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:20.996108    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2836af7a-f094-4a6f-bbb7-62ae5d544374-maas-api-tls\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:21.002353 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:21.002326    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plljq\" (UniqueName: \"kubernetes.io/projected/2836af7a-f094-4a6f-bbb7-62ae5d544374-kube-api-access-plljq\") pod \"maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") " pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:21.101534 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:21.101483    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:21.238440 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:21.238409    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"]
Jul 07 02:45:21.241411 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:45:21.241383    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2836af7a_f094_4a6f_bbb7_62ae5d544374.slice/crio-0139930044dc2709eb71827ca657c9dfe71ed355377c93a1f396755f54ebf7fb WatchSource:0}: Error finding container 0139930044dc2709eb71827ca657c9dfe71ed355377c93a1f396755f54ebf7fb: Status 404 returned error can't find the container with id 0139930044dc2709eb71827ca657c9dfe71ed355377c93a1f396755f54ebf7fb
Jul 07 02:45:21.381974 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:21.381938    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:21.483106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:21.483056    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:21.514363 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:21.514286    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z" event={"ID":"2836af7a-f094-4a6f-bbb7-62ae5d544374","Type":"ContainerStarted","Data":"0139930044dc2709eb71827ca657c9dfe71ed355377c93a1f396755f54ebf7fb"}
Jul 07 02:45:22.520165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:22.520122    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z" event={"ID":"2836af7a-f094-4a6f-bbb7-62ae5d544374","Type":"ContainerStarted","Data":"be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb"}
Jul 07 02:45:22.520570 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:22.520192    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:22.538981 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:22.538932    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z" podStartSLOduration=2.065684601 podStartE2EDuration="2.53891416s" totalImagesPullingTime="473.229559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:45:22.536618622 +0000 UTC m=+2132.740039367" watchObservedRunningTime="2026-07-07 02:45:22.53891416 +0000 UTC m=+2132.742334903"
Jul 07 02:45:25.130647 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:25.130614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:25.598360 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:25.598329    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:25.917454 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:25.917328    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:26.309559 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:26.309513    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:27.428871 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:27.428829    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:45:28.530080 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.530055    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:28.749673 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.749621    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:28.806728 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.806635    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"]
Jul 07 02:45:28.810452 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.810427    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:28.814118 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.814096    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-833817-serving-cert\""
Jul 07 02:45:28.823776 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.823750    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"]
Jul 07 02:45:28.854608 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.854573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:28.943338 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.943310    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:28.972795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.972760    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-maas-api-tls\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:28.972795 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.972795    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-openshift-service-ca\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:28.972990 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:28.972896    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pph7w\" (UniqueName: \"kubernetes.io/projected/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-kube-api-access-pph7w\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.039565 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.039525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:29.073701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.073605    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-maas-api-tls\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.073701 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.073657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-openshift-service-ca\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.073878 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.073764    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pph7w\" (UniqueName: \"kubernetes.io/projected/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-kube-api-access-pph7w\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.074353 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.074326    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-openshift-service-ca\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.076090 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.076072    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-maas-api-tls\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.082750 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.082728    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pph7w\" (UniqueName: \"kubernetes.io/projected/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-kube-api-access-pph7w\") pod \"maas-api-e2e-api-b-833817-7bb777669c-f2k7k\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") " pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.124799 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.124777    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:29.268323 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.268293    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"]
Jul 07 02:45:29.269351 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:45:29.269317    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59227fe5_b7d5_410b_8e1f_d5e3fe365c17.slice/crio-2ff55c5dd45615ba799627b25ba0d579cee263fc35f1f5e6556c71c31cf7c4e2 WatchSource:0}: Error finding container 2ff55c5dd45615ba799627b25ba0d579cee263fc35f1f5e6556c71c31cf7c4e2: Status 404 returned error can't find the container with id 2ff55c5dd45615ba799627b25ba0d579cee263fc35f1f5e6556c71c31cf7c4e2
Jul 07 02:45:29.400035 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.399940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:29.501187 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.501156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:29.555455 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:29.555422    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k" event={"ID":"59227fe5-b7d5-410b-8e1f-d5e3fe365c17","Type":"ContainerStarted","Data":"2ff55c5dd45615ba799627b25ba0d579cee263fc35f1f5e6556c71c31cf7c4e2"}
Jul 07 02:45:30.561689 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:30.561653    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k" event={"ID":"59227fe5-b7d5-410b-8e1f-d5e3fe365c17","Type":"ContainerStarted","Data":"de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831"}
Jul 07 02:45:30.562104 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:30.561777    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:30.580184 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:30.580136    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k" podStartSLOduration=2.132022462 podStartE2EDuration="2.580119007s" totalImagesPullingTime="448.096545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:45:30.578220078 +0000 UTC m=+2140.781640821" watchObservedRunningTime="2026-07-07 02:45:30.580119007 +0000 UTC m=+2140.783539750"
Jul 07 02:45:33.165171 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:33.165130    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:36.579581 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:36.579548    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:38.226151 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:38.226115    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:45:41.226134 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:41.226100    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:41.684271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:41.684235    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:41.871231 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:41.871198    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:41.976636 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:41.976555    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:42.107007 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.106978    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"]
Jul 07 02:45:42.107209 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.107184    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z" podUID="2836af7a-f094-4a6f-bbb7-62ae5d544374" containerName="maas-api" containerID="cri-o://be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb" gracePeriod=30
Jul 07 02:45:42.136033 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.136009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:42.353751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.353730    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:42.403312 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.403277    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2836af7a-f094-4a6f-bbb7-62ae5d544374-openshift-service-ca\") pod \"2836af7a-f094-4a6f-bbb7-62ae5d544374\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") "
Jul 07 02:45:42.403480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.403346    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plljq\" (UniqueName: \"kubernetes.io/projected/2836af7a-f094-4a6f-bbb7-62ae5d544374-kube-api-access-plljq\") pod \"2836af7a-f094-4a6f-bbb7-62ae5d544374\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") "
Jul 07 02:45:42.403480 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.403387    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2836af7a-f094-4a6f-bbb7-62ae5d544374-maas-api-tls\") pod \"2836af7a-f094-4a6f-bbb7-62ae5d544374\" (UID: \"2836af7a-f094-4a6f-bbb7-62ae5d544374\") "
Jul 07 02:45:42.403729 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.403701    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2836af7a-f094-4a6f-bbb7-62ae5d544374-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2836af7a-f094-4a6f-bbb7-62ae5d544374" (UID: "2836af7a-f094-4a6f-bbb7-62ae5d544374"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:45:42.405545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.405480    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2836af7a-f094-4a6f-bbb7-62ae5d544374-kube-api-access-plljq" (OuterVolumeSpecName: "kube-api-access-plljq") pod "2836af7a-f094-4a6f-bbb7-62ae5d544374" (UID: "2836af7a-f094-4a6f-bbb7-62ae5d544374"). InnerVolumeSpecName "kube-api-access-plljq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:45:42.405667 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.405609    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2836af7a-f094-4a6f-bbb7-62ae5d544374-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2836af7a-f094-4a6f-bbb7-62ae5d544374" (UID: "2836af7a-f094-4a6f-bbb7-62ae5d544374"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:45:42.504617 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.504588    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2836af7a-f094-4a6f-bbb7-62ae5d544374-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:45:42.504617 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.504611    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plljq\" (UniqueName: \"kubernetes.io/projected/2836af7a-f094-4a6f-bbb7-62ae5d544374-kube-api-access-plljq\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:45:42.504617 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.504620    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2836af7a-f094-4a6f-bbb7-62ae5d544374-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:45:42.623298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.623204    2568 generic.go:356] "Generic (PLEG): container finished" podID="2836af7a-f094-4a6f-bbb7-62ae5d544374" containerID="be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb" exitCode=0
Jul 07 02:45:42.623298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.623239    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z" event={"ID":"2836af7a-f094-4a6f-bbb7-62ae5d544374","Type":"ContainerDied","Data":"be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb"}
Jul 07 02:45:42.623298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.623292    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z" event={"ID":"2836af7a-f094-4a6f-bbb7-62ae5d544374","Type":"ContainerDied","Data":"0139930044dc2709eb71827ca657c9dfe71ed355377c93a1f396755f54ebf7fb"}
Jul 07 02:45:42.623562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.623315    2568 scope.go:117] "RemoveContainer" containerID="be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb"
Jul 07 02:45:42.623562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.623250    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"
Jul 07 02:45:42.631877 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.631857    2568 scope.go:117] "RemoveContainer" containerID="be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb"
Jul 07 02:45:42.632108 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:45:42.632091    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb\": container with ID starting with be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb not found: ID does not exist" containerID="be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb"
Jul 07 02:45:42.632152 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.632117    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb"} err="failed to get container status \"be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb\": rpc error: code = NotFound desc = could not find container \"be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb\": container with ID starting with be50f580cfa7d15456d8418fb96c78a2dd9f8d9069c7c83565ad6deca91704cb not found: ID does not exist"
Jul 07 02:45:42.641535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.641514    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"]
Jul 07 02:45:42.645711 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:42.645689    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8ad891-7bdc6466b9-95v5z"]
Jul 07 02:45:44.433467 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:44.433439    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2836af7a-f094-4a6f-bbb7-62ae5d544374" path="/var/lib/kubelet/pods/2836af7a-f094-4a6f-bbb7-62ae5d544374/volumes"
Jul 07 02:45:47.482755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:47.482721    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:47.586271 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:47.586227    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:47.758585 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:47.758541    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:48.014461 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.014387    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:48.220996 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.220964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:48.528924 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.528892    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:48.629731 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.629700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:48.729689 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.729659    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:48.844182 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.844099    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"]
Jul 07 02:45:48.844347 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.844308    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k" podUID="59227fe5-b7d5-410b-8e1f-d5e3fe365c17" containerName="maas-api" containerID="cri-o://de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831" gracePeriod=30
Jul 07 02:45:48.888690 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:48.888663    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:49.097330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.097264    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:49.167074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.167043    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pph7w\" (UniqueName: \"kubernetes.io/projected/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-kube-api-access-pph7w\") pod \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") "
Jul 07 02:45:49.167254 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.167120    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-maas-api-tls\") pod \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") "
Jul 07 02:45:49.167254 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.167138    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-openshift-service-ca\") pod \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\" (UID: \"59227fe5-b7d5-410b-8e1f-d5e3fe365c17\") "
Jul 07 02:45:49.167572 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.167549    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59227fe5-b7d5-410b-8e1f-d5e3fe365c17" (UID: "59227fe5-b7d5-410b-8e1f-d5e3fe365c17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:45:49.169178 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.169151    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59227fe5-b7d5-410b-8e1f-d5e3fe365c17" (UID: "59227fe5-b7d5-410b-8e1f-d5e3fe365c17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:45:49.169298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.169222    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-kube-api-access-pph7w" (OuterVolumeSpecName: "kube-api-access-pph7w") pod "59227fe5-b7d5-410b-8e1f-d5e3fe365c17" (UID: "59227fe5-b7d5-410b-8e1f-d5e3fe365c17"). InnerVolumeSpecName "kube-api-access-pph7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:45:49.268373 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.268327    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pph7w\" (UniqueName: \"kubernetes.io/projected/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-kube-api-access-pph7w\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:45:49.268373 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.268356    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:45:49.268373 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.268366    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59227fe5-b7d5-410b-8e1f-d5e3fe365c17-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:45:49.659155 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.659124    2568 generic.go:356] "Generic (PLEG): container finished" podID="59227fe5-b7d5-410b-8e1f-d5e3fe365c17" containerID="de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831" exitCode=0
Jul 07 02:45:49.659609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.659168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k" event={"ID":"59227fe5-b7d5-410b-8e1f-d5e3fe365c17","Type":"ContainerDied","Data":"de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831"}
Jul 07 02:45:49.659609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.659196    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"
Jul 07 02:45:49.659609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.659210    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k" event={"ID":"59227fe5-b7d5-410b-8e1f-d5e3fe365c17","Type":"ContainerDied","Data":"2ff55c5dd45615ba799627b25ba0d579cee263fc35f1f5e6556c71c31cf7c4e2"}
Jul 07 02:45:49.659609 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.659230    2568 scope.go:117] "RemoveContainer" containerID="de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831"
Jul 07 02:45:49.676504 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.676471    2568 scope.go:117] "RemoveContainer" containerID="de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831"
Jul 07 02:45:49.676974 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:45:49.676946    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831\": container with ID starting with de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831 not found: ID does not exist" containerID="de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831"
Jul 07 02:45:49.677081 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.676983    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831"} err="failed to get container status \"de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831\": rpc error: code = NotFound desc = could not find container \"de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831\": container with ID starting with de3eb1562f1744aa2a2472e6b0f665976ce03aa1090d4abda5482c82a445d831 not found: ID does not exist"
Jul 07 02:45:49.688044 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.688018    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"]
Jul 07 02:45:49.690485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:49.690467    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-833817-7bb777669c-f2k7k"]
Jul 07 02:45:50.433659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:50.433630    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59227fe5-b7d5-410b-8e1f-d5e3fe365c17" path="/var/lib/kubelet/pods/59227fe5-b7d5-410b-8e1f-d5e3fe365c17/volumes"
Jul 07 02:45:51.719856 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:51.719823    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:45:54.583845 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:54.583812    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:54.684093 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:54.684062    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:54.843893 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:54.843806    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:55.056521 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:55.056470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:55.385253 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:55.385219    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:55.699214 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:55.699133    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:56.067317 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:56.067283    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:58.928723 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:58.928687    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:59.020732 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:59.020693    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:59.123940 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:59.123910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:59.339211 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:59.339176    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:59.488115 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:59.488084    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:45:59.596567 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:45:59.596462    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:03.414679 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:03.414642    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:03.757891 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:03.757862    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:04.047062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:04.046976    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:04.664894 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:04.664863    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:06.018122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:06.018086    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:46:06.986134 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:06.986101    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:07.092204 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.092174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:07.133608 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.133580    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"]
Jul 07 02:46:07.133968 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.133955    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59227fe5-b7d5-410b-8e1f-d5e3fe365c17" containerName="maas-api"
Jul 07 02:46:07.134034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.133969    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="59227fe5-b7d5-410b-8e1f-d5e3fe365c17" containerName="maas-api"
Jul 07 02:46:07.134034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.133985    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2836af7a-f094-4a6f-bbb7-62ae5d544374" containerName="maas-api"
Jul 07 02:46:07.134034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.133990    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2836af7a-f094-4a6f-bbb7-62ae5d544374" containerName="maas-api"
Jul 07 02:46:07.134183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.134064    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2836af7a-f094-4a6f-bbb7-62ae5d544374" containerName="maas-api"
Jul 07 02:46:07.134183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.134074    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="59227fe5-b7d5-410b-8e1f-d5e3fe365c17" containerName="maas-api"
Jul 07 02:46:07.137260 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.137237    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.139943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.139918    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-3dabf0-serving-cert\""
Jul 07 02:46:07.140085 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.139972    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\""
Jul 07 02:46:07.147094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.147072    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"]
Jul 07 02:46:07.147547 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.147484    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26j4m\" (UniqueName: \"kubernetes.io/projected/1422ba11-1d45-4536-a833-093459bf686e-kube-api-access-26j4m\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.147769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.147736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1422ba11-1d45-4536-a833-093459bf686e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.147873 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.147798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.185763 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.185722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:07.248896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.248862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26j4m\" (UniqueName: \"kubernetes.io/projected/1422ba11-1d45-4536-a833-093459bf686e-kube-api-access-26j4m\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.249097 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.248909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1422ba11-1d45-4536-a833-093459bf686e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.249097 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.248937    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.249097 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:46:07.249039    2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-3dabf0-serving-cert: secret "maas-api-e2e-shared-b-3dabf0-serving-cert" not found
Jul 07 02:46:07.249268 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:46:07.249100    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls podName:1422ba11-1d45-4536-a833-093459bf686e nodeName:}" failed. No retries permitted until 2026-07-07 02:46:07.749081248 +0000 UTC m=+2177.952501970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls") pod "maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" (UID: "1422ba11-1d45-4536-a833-093459bf686e") : secret "maas-api-e2e-shared-b-3dabf0-serving-cert" not found
Jul 07 02:46:07.249687 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.249665    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1422ba11-1d45-4536-a833-093459bf686e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.260282 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.260251    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26j4m\" (UniqueName: \"kubernetes.io/projected/1422ba11-1d45-4536-a833-093459bf686e-kube-api-access-26j4m\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.627948 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.627849    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:07.728862 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.728828    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:07.754824 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:07.754789    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:07.755009 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:46:07.754936    2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-3dabf0-serving-cert: secret "maas-api-e2e-shared-b-3dabf0-serving-cert" not found
Jul 07 02:46:07.755009 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:46:07.755007    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls podName:1422ba11-1d45-4536-a833-093459bf686e nodeName:}" failed. No retries permitted until 2026-07-07 02:46:08.754991716 +0000 UTC m=+2178.958412438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls") pod "maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" (UID: "1422ba11-1d45-4536-a833-093459bf686e") : secret "maas-api-e2e-shared-b-3dabf0-serving-cert" not found
Jul 07 02:46:08.024713 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:08.024677    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:08.332666 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:08.332553    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:08.460789 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:08.460745    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:08.765235 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:08.765196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:08.767861 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:08.767832    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") pod \"maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") " pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:08.950844 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:08.950808    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:09.099335 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.099299    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:09.190693 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.190650    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:09.296663 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.296605    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"]
Jul 07 02:46:09.298936 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:46:09.298909    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1422ba11_1d45_4536_a833_093459bf686e.slice/crio-2c82b137243527b691670ad4254d36a508efb7628e3ffc0424e6f3fdbe65e90f WatchSource:0}: Error finding container 2c82b137243527b691670ad4254d36a508efb7628e3ffc0424e6f3fdbe65e90f: Status 404 returned error can't find the container with id 2c82b137243527b691670ad4254d36a508efb7628e3ffc0424e6f3fdbe65e90f
Jul 07 02:46:09.758197 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.758156    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" event={"ID":"1422ba11-1d45-4536-a833-093459bf686e","Type":"ContainerStarted","Data":"482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48"}
Jul 07 02:46:09.758579 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.758220    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" event={"ID":"1422ba11-1d45-4536-a833-093459bf686e","Type":"ContainerStarted","Data":"2c82b137243527b691670ad4254d36a508efb7628e3ffc0424e6f3fdbe65e90f"}
Jul 07 02:46:09.758579 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.758375    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:09.775716 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:09.775147    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" podStartSLOduration=2.393733099 podStartE2EDuration="2.775125191s" totalImagesPullingTime="381.392092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:46:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:46:09.774217861 +0000 UTC m=+2179.977638629" watchObservedRunningTime="2026-07-07 02:46:09.775125191 +0000 UTC m=+2179.978545935"
Jul 07 02:46:11.452624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:11.452584    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:15.770298 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:15.770271    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:46:16.915417 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:16.915379    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:46:17.026217 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.026177    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"]
Jul 07 02:46:17.030071 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.030048    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.032443 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.032423    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f01161\"/\"kube-root-ca.crt\""
Jul 07 02:46:17.033345 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.033326    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f01161\"/\"openshift-service-ca.crt\""
Jul 07 02:46:17.033457 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.033352    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f01161\"/\"auth-test-model-f01161-kserve-self-signed-certs\""
Jul 07 02:46:17.033457 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.033365    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f01161\"/\"default-dockercfg-ssn9v\""
Jul 07 02:46:17.038880 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.038859    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"]
Jul 07 02:46:17.123468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.123391    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:17.141439 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.141405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjrl9\" (UniqueName: \"kubernetes.io/projected/79a549fd-2670-49d7-89ce-be6c040de952-kube-api-access-fjrl9\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.141618 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.141529    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a549fd-2670-49d7-89ce-be6c040de952-tls-certs\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.141618 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.141578    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-dshm\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.141693 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.141634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-model-cache\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.141693 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.141652    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-home\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.141693 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.141680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-tmp-dir\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243008 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.242975    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-model-cache\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243014    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-home\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243042    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-tmp-dir\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjrl9\" (UniqueName: \"kubernetes.io/projected/79a549fd-2670-49d7-89ce-be6c040de952-kube-api-access-fjrl9\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243198 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243182    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a549fd-2670-49d7-89ce-be6c040de952-tls-certs\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243441 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243211    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-dshm\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243484    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-tmp-dir\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243628 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243582    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-home\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.243687 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.243641    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-model-cache\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.245400 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.245374    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-dshm\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.245892 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.245872    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a549fd-2670-49d7-89ce-be6c040de952-tls-certs\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.251922 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.251895    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjrl9\" (UniqueName: \"kubernetes.io/projected/79a549fd-2670-49d7-89ce-be6c040de952-kube-api-access-fjrl9\") pod \"auth-test-model-f01161-kserve-78f9c94d6c-ljslh\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") " pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.340031 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.339998    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:17.481259 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.481235    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"]
Jul 07 02:46:17.483463 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:46:17.483430    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79a549fd_2670_49d7_89ce_be6c040de952.slice/crio-3d7b931e0d075cb5d99492ba9f3bed82710323c2d65ec7a19325fae5924f33f4 WatchSource:0}: Error finding container 3d7b931e0d075cb5d99492ba9f3bed82710323c2d65ec7a19325fae5924f33f4: Status 404 returned error can't find the container with id 3d7b931e0d075cb5d99492ba9f3bed82710323c2d65ec7a19325fae5924f33f4
Jul 07 02:46:17.803688 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.803597    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh" event={"ID":"79a549fd-2670-49d7-89ce-be6c040de952","Type":"ContainerStarted","Data":"e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6"}
Jul 07 02:46:17.803688 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.803643    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh" event={"ID":"79a549fd-2670-49d7-89ce-be6c040de952","Type":"ContainerStarted","Data":"3d7b931e0d075cb5d99492ba9f3bed82710323c2d65ec7a19325fae5924f33f4"}
Jul 07 02:46:17.821184 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:17.821135    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh" podStartSLOduration=0.82112109 podStartE2EDuration="821.12109ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:46:17.819781397 +0000 UTC m=+2188.023202143" watchObservedRunningTime="2026-07-07 02:46:17.82112109 +0000 UTC m=+2188.024541835"
Jul 07 02:46:20.330179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:20.330144    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:20.431545 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:20.431508    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:26.227434 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:26.227401    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:46:27.340562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:27.340531    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:27.340951 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:27.340582    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:27.353054 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:27.353031    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:27.863756 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:27.863727    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:46:28.328748 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:28.328714    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:28.434339 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:28.434306    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:40.932551 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.932516    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"]
Jul 07 02:46:40.936314 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.936289    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:40.939809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.939783    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"openshift-service-ca.crt\""
Jul 07 02:46:40.939967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.939845    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"default-dockercfg-lcrs5\""
Jul 07 02:46:40.939967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.939846    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"kube-root-ca.crt\""
Jul 07 02:46:40.939967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.939937    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"auth-test-model-3dabf0-kserve-self-signed-certs\""
Jul 07 02:46:40.949226 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.949199    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"]
Jul 07 02:46:40.957751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:40.957724    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:46:41.077814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.077774    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvkpt\" (UniqueName: \"kubernetes.io/projected/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-kube-api-access-vvkpt\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.077994 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.077826    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-home\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.077994 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.077856    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-model-cache\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.077994 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.077873    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tls-certs\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.077994 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.077911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tmp-dir\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.077994 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.077973    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-dshm\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.178975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.178941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-home\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.178975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.178977    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-model-cache\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179249 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.178995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tls-certs\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179249 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.179050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tmp-dir\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179249 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.179096    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-dshm\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179249 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.179168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvkpt\" (UniqueName: \"kubernetes.io/projected/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-kube-api-access-vvkpt\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179461 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.179440    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-home\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179461 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.179452    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-model-cache\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.179643 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.179626    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tmp-dir\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.181371 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.181347    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-dshm\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.181691 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.181676    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tls-certs\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.188062 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.187996    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvkpt\" (UniqueName: \"kubernetes.io/projected/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-kube-api-access-vvkpt\") pod \"auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") " pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.248465 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.248429    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:41.395183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.395155    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"]
Jul 07 02:46:41.398608 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:46:41.398576    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod563a7eb1_b94a_4106_adbb_9eebfbd8c3e7.slice/crio-f143055c927a0c18d7fab14ab4ca5d568279730c9e93183371fbbcfe2ff36e9b WatchSource:0}: Error finding container f143055c927a0c18d7fab14ab4ca5d568279730c9e93183371fbbcfe2ff36e9b: Status 404 returned error can't find the container with id f143055c927a0c18d7fab14ab4ca5d568279730c9e93183371fbbcfe2ff36e9b
Jul 07 02:46:41.931786 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.931752    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b" event={"ID":"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7","Type":"ContainerStarted","Data":"2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09"}
Jul 07 02:46:41.931786 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.931791    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b" event={"ID":"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7","Type":"ContainerStarted","Data":"f143055c927a0c18d7fab14ab4ca5d568279730c9e93183371fbbcfe2ff36e9b"}
Jul 07 02:46:41.950946 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:41.950895    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b" podStartSLOduration=1.950881732 podStartE2EDuration="1.950881732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:46:41.948872597 +0000 UTC m=+2212.152293343" watchObservedRunningTime="2026-07-07 02:46:41.950881732 +0000 UTC m=+2212.154302475"
Jul 07 02:46:44.319594 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:44.319554    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:46:51.249196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:51.249152    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:51.249196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:51.249206    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:51.262864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:51.262835    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:51.993284 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:51.993254    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:46:56.919431 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:46:56.919393    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:47:03.811150 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:47:03.811114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:47:03.871162 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:47:03.871122    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:47:11.420594 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:47:11.420556    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:47:24.010694 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:47:24.010654    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:47:40.220046 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:47:40.220010    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:47:56.127153 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:47:56.127104    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:48:12.121825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:12.121787    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:48:20.240802 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:20.240770    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:48:29.230468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:29.230438    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:48:33.267310 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:33.267271    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"]
Jul 07 02:48:33.285244 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:33.285212    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:48:34.106960 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:34.106918    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"]
Jul 07 02:48:34.129227 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:34.129200    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:48:38.237439 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.237405    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"]
Jul 07 02:48:38.240824 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.240806    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.244083 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.244055    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f01161\"/\"sub-test-model-f01161-kserve-self-signed-certs\""
Jul 07 02:48:38.259106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.259067    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"]
Jul 07 02:48:38.270349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.270319    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:48:38.384774 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.384738    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-tmp-dir\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.384952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.384795    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3847a5de-66bb-4148-8705-2867f2363298-tls-certs\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.384952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.384875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-home\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.384952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.384893    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-dshm\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.384952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.384914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c9r7\" (UniqueName: \"kubernetes.io/projected/3847a5de-66bb-4148-8705-2867f2363298-kube-api-access-9c9r7\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.384952 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.384950    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-model-cache\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486069 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c9r7\" (UniqueName: \"kubernetes.io/projected/3847a5de-66bb-4148-8705-2867f2363298-kube-api-access-9c9r7\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486255 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486077    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-model-cache\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486255 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-tmp-dir\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486255 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3847a5de-66bb-4148-8705-2867f2363298-tls-certs\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486255 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486211    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-home\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486255 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486225    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-dshm\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486622 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486600    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-model-cache\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486691 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486626    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-tmp-dir\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.486732 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.486690    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-home\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.488626 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.488567    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-dshm\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.488807 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.488792    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3847a5de-66bb-4148-8705-2867f2363298-tls-certs\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.495477 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.495444    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c9r7\" (UniqueName: \"kubernetes.io/projected/3847a5de-66bb-4148-8705-2867f2363298-kube-api-access-9c9r7\") pod \"sub-test-model-f01161-kserve-6564c594fc-cmfhq\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") " pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.550924 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.550874    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:38.695650 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:38.695625    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"]
Jul 07 02:48:38.698664 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:48:38.698634    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3847a5de_66bb_4148_8705_2867f2363298.slice/crio-ce6f5df8ab791b2cf444585cb989094d5bfedbf3e4939a3a9a8a00cfe2973f1f WatchSource:0}: Error finding container ce6f5df8ab791b2cf444585cb989094d5bfedbf3e4939a3a9a8a00cfe2973f1f: Status 404 returned error can't find the container with id ce6f5df8ab791b2cf444585cb989094d5bfedbf3e4939a3a9a8a00cfe2973f1f
Jul 07 02:48:39.522183 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:39.522149    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq" event={"ID":"3847a5de-66bb-4148-8705-2867f2363298","Type":"ContainerStarted","Data":"9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c"}
Jul 07 02:48:39.522605 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:39.522193    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq" event={"ID":"3847a5de-66bb-4148-8705-2867f2363298","Type":"ContainerStarted","Data":"ce6f5df8ab791b2cf444585cb989094d5bfedbf3e4939a3a9a8a00cfe2973f1f"}
Jul 07 02:48:39.539934 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:39.539880    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq" podStartSLOduration=1.53986162 podStartE2EDuration="1.53986162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:48:39.537845447 +0000 UTC m=+2329.741266194" watchObservedRunningTime="2026-07-07 02:48:39.53986162 +0000 UTC m=+2329.743282361"
Jul 07 02:48:46.422658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:46.422622    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:48:48.287759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.287698    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh" podUID="79a549fd-2670-49d7-89ce-be6c040de952" containerName="main" containerID="cri-o://e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6" gracePeriod=45
Jul 07 02:48:48.528413 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.528387    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:48:48.551977 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.551898    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:48.551977 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.551953    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:48.569873 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.569848    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:48.570516 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.570470    2568 generic.go:356] "Generic (PLEG): container finished" podID="79a549fd-2670-49d7-89ce-be6c040de952" containerID="e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6" exitCode=0
Jul 07 02:48:48.570658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.570539    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"
Jul 07 02:48:48.570658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.570539    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh" event={"ID":"79a549fd-2670-49d7-89ce-be6c040de952","Type":"ContainerDied","Data":"e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6"}
Jul 07 02:48:48.570658 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.570644    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh" event={"ID":"79a549fd-2670-49d7-89ce-be6c040de952","Type":"ContainerDied","Data":"3d7b931e0d075cb5d99492ba9f3bed82710323c2d65ec7a19325fae5924f33f4"}
Jul 07 02:48:48.570825 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.570668    2568 scope.go:117] "RemoveContainer" containerID="e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6"
Jul 07 02:48:48.581098 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.581075    2568 scope.go:117] "RemoveContainer" containerID="e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6"
Jul 07 02:48:48.581383 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:48:48.581357    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6\": container with ID starting with e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6 not found: ID does not exist" containerID="e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6"
Jul 07 02:48:48.581529 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.581389    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6"} err="failed to get container status \"e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6\": rpc error: code = NotFound desc = could not find container \"e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6\": container with ID starting with e2883dfe9bf12daed2b93c22ad4162c8d978e34c7536f3136f22d6a54587ead6 not found: ID does not exist"
Jul 07 02:48:48.584927 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.584904    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:48:48.689587 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689552    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a549fd-2670-49d7-89ce-be6c040de952-tls-certs\") pod \"79a549fd-2670-49d7-89ce-be6c040de952\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") "
Jul 07 02:48:48.689755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689596    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjrl9\" (UniqueName: \"kubernetes.io/projected/79a549fd-2670-49d7-89ce-be6c040de952-kube-api-access-fjrl9\") pod \"79a549fd-2670-49d7-89ce-be6c040de952\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") "
Jul 07 02:48:48.689755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689636    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-tmp-dir\") pod \"79a549fd-2670-49d7-89ce-be6c040de952\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") "
Jul 07 02:48:48.689755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689672    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-home\") pod \"79a549fd-2670-49d7-89ce-be6c040de952\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") "
Jul 07 02:48:48.689755 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689734    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-model-cache\") pod \"79a549fd-2670-49d7-89ce-be6c040de952\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") "
Jul 07 02:48:48.689967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689759    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-dshm\") pod \"79a549fd-2670-49d7-89ce-be6c040de952\" (UID: \"79a549fd-2670-49d7-89ce-be6c040de952\") "
Jul 07 02:48:48.690024 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.689979    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79a549fd-2670-49d7-89ce-be6c040de952" (UID: "79a549fd-2670-49d7-89ce-be6c040de952"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:48.690079 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.690064    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-model-cache" (OuterVolumeSpecName: "model-cache") pod "79a549fd-2670-49d7-89ce-be6c040de952" (UID: "79a549fd-2670-49d7-89ce-be6c040de952"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:48.690131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.690082    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-home" (OuterVolumeSpecName: "home") pod "79a549fd-2670-49d7-89ce-be6c040de952" (UID: "79a549fd-2670-49d7-89ce-be6c040de952"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:48.690176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.690149    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:48.690176 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.690166    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:48.691938 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.691907    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79a549fd-2670-49d7-89ce-be6c040de952-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79a549fd-2670-49d7-89ce-be6c040de952" (UID: "79a549fd-2670-49d7-89ce-be6c040de952"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:48:48.692055 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.692005    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a549fd-2670-49d7-89ce-be6c040de952-kube-api-access-fjrl9" (OuterVolumeSpecName: "kube-api-access-fjrl9") pod "79a549fd-2670-49d7-89ce-be6c040de952" (UID: "79a549fd-2670-49d7-89ce-be6c040de952"). InnerVolumeSpecName "kube-api-access-fjrl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:48:48.692109 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.692050    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-dshm" (OuterVolumeSpecName: "dshm") pod "79a549fd-2670-49d7-89ce-be6c040de952" (UID: "79a549fd-2670-49d7-89ce-be6c040de952"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:48.791428 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.791401    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a549fd-2670-49d7-89ce-be6c040de952-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:48.791428 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.791424    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjrl9\" (UniqueName: \"kubernetes.io/projected/79a549fd-2670-49d7-89ce-be6c040de952-kube-api-access-fjrl9\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:48.791428 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.791433    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:48.791659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.791442    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a549fd-2670-49d7-89ce-be6c040de952-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:48.892780 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.892752    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"]
Jul 07 02:48:48.898903 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:48.898877    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/auth-test-model-f01161-kserve-78f9c94d6c-ljslh"]
Jul 07 02:48:49.127722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.127678    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b" podUID="563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" containerName="main" containerID="cri-o://2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09" gracePeriod=45
Jul 07 02:48:49.384769 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.384715    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:48:49.500436 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500404    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-home\") pod \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") "
Jul 07 02:48:49.500618 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500453    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-model-cache\") pod \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") "
Jul 07 02:48:49.500618 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500522    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tmp-dir\") pod \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") "
Jul 07 02:48:49.500618 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500579    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvkpt\" (UniqueName: \"kubernetes.io/projected/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-kube-api-access-vvkpt\") pod \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") "
Jul 07 02:48:49.500749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500618    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tls-certs\") pod \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") "
Jul 07 02:48:49.500749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500660    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-dshm\") pod \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\" (UID: \"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7\") "
Jul 07 02:48:49.500749 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500722    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-home" (OuterVolumeSpecName: "home") pod "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" (UID: "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:49.500891 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500787    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" (UID: "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:49.500891 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.500846    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" (UID: "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:49.501106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.501081    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:49.501106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.501105    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:49.501106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.501113    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:49.502910 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.502889    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-dshm" (OuterVolumeSpecName: "dshm") pod "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" (UID: "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:48:49.502993 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.502935    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-kube-api-access-vvkpt" (OuterVolumeSpecName: "kube-api-access-vvkpt") pod "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" (UID: "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7"). InnerVolumeSpecName "kube-api-access-vvkpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:48:49.503043 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.502997    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" (UID: "563a7eb1-b94a-4106-adbb-9eebfbd8c3e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:48:49.578122 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.578092    2568 generic.go:356] "Generic (PLEG): container finished" podID="563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" containerID="2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09" exitCode=0
Jul 07 02:48:49.578277 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.578141    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"
Jul 07 02:48:49.578277 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.578184    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b" event={"ID":"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7","Type":"ContainerDied","Data":"2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09"}
Jul 07 02:48:49.578277 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.578223    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b" event={"ID":"563a7eb1-b94a-4106-adbb-9eebfbd8c3e7","Type":"ContainerDied","Data":"f143055c927a0c18d7fab14ab4ca5d568279730c9e93183371fbbcfe2ff36e9b"}
Jul 07 02:48:49.578277 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.578248    2568 scope.go:117] "RemoveContainer" containerID="2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09"
Jul 07 02:48:49.587861 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.587841    2568 scope.go:117] "RemoveContainer" containerID="2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09"
Jul 07 02:48:49.588150 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:48:49.588129    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09\": container with ID starting with 2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09 not found: ID does not exist" containerID="2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09"
Jul 07 02:48:49.588199 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.588158    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09"} err="failed to get container status \"2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09\": rpc error: code = NotFound desc = could not find container \"2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09\": container with ID starting with 2c4ffcad78ecc472261cb7ea7b30cdd330b12f9ece5212814d9b2403b610fb09 not found: ID does not exist"
Jul 07 02:48:49.600105 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.600082    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"]
Jul 07 02:48:49.602526 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.602479    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvkpt\" (UniqueName: \"kubernetes.io/projected/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-kube-api-access-vvkpt\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:49.602526 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.602522    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:49.602670 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.602534    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:48:49.606084 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:49.606062    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/auth-test-model-3dabf0-kserve-77b94d6f47-dsf9b"]
Jul 07 02:48:50.433696 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:50.433667    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" path="/var/lib/kubelet/pods/563a7eb1-b94a-4106-adbb-9eebfbd8c3e7/volumes"
Jul 07 02:48:50.434078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:48:50.434064    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79a549fd-2670-49d7-89ce-be6c040de952" path="/var/lib/kubelet/pods/79a549fd-2670-49d7-89ce-be6c040de952/volumes"
Jul 07 02:49:15.830466 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.830427    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"]
Jul 07 02:49:15.831023 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.830954    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" containerName="main"
Jul 07 02:49:15.831023 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.830970    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" containerName="main"
Jul 07 02:49:15.831023 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.830999    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a549fd-2670-49d7-89ce-be6c040de952" containerName="main"
Jul 07 02:49:15.831023 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.831004    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a549fd-2670-49d7-89ce-be6c040de952" containerName="main"
Jul 07 02:49:15.831173 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.831067    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="563a7eb1-b94a-4106-adbb-9eebfbd8c3e7" containerName="main"
Jul 07 02:49:15.831173 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.831081    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="79a549fd-2670-49d7-89ce-be6c040de952" containerName="main"
Jul 07 02:49:15.834188 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.834172    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:15.837704 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.837683    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"kube-root-ca.crt\""
Jul 07 02:49:15.837833 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.837686    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"sub-test-model-3dabf0-kserve-self-signed-certs\""
Jul 07 02:49:15.837833 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.837767    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"default-dockercfg-lcrs5\""
Jul 07 02:49:15.837833 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.837731    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"openshift-service-ca.crt\""
Jul 07 02:49:15.841844 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.841826    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"]
Jul 07 02:49:15.925132 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.925098    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:49:15.947167 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.947133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4rjt\" (UniqueName: \"kubernetes.io/projected/bf4c5796-415c-4fe2-b491-c03961ff3775-kube-api-access-x4rjt\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:15.947319 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.947171    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-home\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:15.947319 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.947235    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-tmp-dir\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:15.947408 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.947313    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-dshm\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:15.947408 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.947351    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-model-cache\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:15.947408 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:15.947370    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4c5796-415c-4fe2-b491-c03961ff3775-tls-certs\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.048478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048440    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4rjt\" (UniqueName: \"kubernetes.io/projected/bf4c5796-415c-4fe2-b491-c03961ff3775-kube-api-access-x4rjt\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.048478 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-home\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.048722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-tmp-dir\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.048722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048576    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-dshm\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.048722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-model-cache\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.048722 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4c5796-415c-4fe2-b491-c03961ff3775-tls-certs\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.049016 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.048991    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-home\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.049063 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.049002    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-tmp-dir\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.049114 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.049096    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-model-cache\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.050975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.050956    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-dshm\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.051131 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.051115    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4c5796-415c-4fe2-b491-c03961ff3775-tls-certs\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.057608 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.057587    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4rjt\" (UniqueName: \"kubernetes.io/projected/bf4c5796-415c-4fe2-b491-c03961ff3775-kube-api-access-x4rjt\") pod \"sub-test-model-3dabf0-kserve-5f69877c87-w8nfq\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") " pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.143743 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.143653    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:16.278668 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.278640    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"]
Jul 07 02:49:16.281315 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:49:16.281285    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf4c5796_415c_4fe2_b491_c03961ff3775.slice/crio-d168613ab19160a9c68c303f4211f32331e614f501152128f5f767aeab0c766c WatchSource:0}: Error finding container d168613ab19160a9c68c303f4211f32331e614f501152128f5f767aeab0c766c: Status 404 returned error can't find the container with id d168613ab19160a9c68c303f4211f32331e614f501152128f5f767aeab0c766c
Jul 07 02:49:16.709752 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.709717    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq" event={"ID":"bf4c5796-415c-4fe2-b491-c03961ff3775","Type":"ContainerStarted","Data":"b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4"}
Jul 07 02:49:16.709939 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.709759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq" event={"ID":"bf4c5796-415c-4fe2-b491-c03961ff3775","Type":"ContainerStarted","Data":"d168613ab19160a9c68c303f4211f32331e614f501152128f5f767aeab0c766c"}
Jul 07 02:49:16.728809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:16.728759    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq" podStartSLOduration=1.728742403 podStartE2EDuration="1.728742403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:49:16.727137554 +0000 UTC m=+2366.930558297" watchObservedRunningTime="2026-07-07 02:49:16.728742403 +0000 UTC m=+2366.932163145"
Jul 07 02:49:19.227040 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:19.227007    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:49:25.877805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:25.877755    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:49:26.144321 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:26.144243    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:26.144321 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:26.144285    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:26.156331 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:26.156313    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:26.768922 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:26.768893    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:49:49.222648 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:49.222618    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:49:49.824326 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:49.824292    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:49:52.641293 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:52.641185    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:49:52.650912 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:49:52.650892    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:50:06.215442 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:06.215403    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:50:11.503017 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:11.502926    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"]
Jul 07 02:50:11.527334 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:11.527231    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:50:12.328659 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:12.328619    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"]
Jul 07 02:50:12.341273 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:12.341244    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:50:18.260328 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:18.260293    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:50:20.116368 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.116327    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"]
Jul 07 02:50:20.119960 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.119942    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.122466 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.122446    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f01161\"/\"rate-test-model-f01161-kserve-self-signed-certs\""
Jul 07 02:50:20.129248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.129226    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"]
Jul 07 02:50:20.139395 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.139375    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:50:20.171367 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.171344    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-tmp-dir\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.171489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.171378    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prrhq\" (UniqueName: \"kubernetes.io/projected/6b04375e-536a-421a-8cf7-d1f57b07fe06-kube-api-access-prrhq\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.171489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.171418    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-model-cache\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.171489 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.171475    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-dshm\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.171625 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.171548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b04375e-536a-421a-8cf7-d1f57b07fe06-tls-certs\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.171625 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.171568    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-home\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272086 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272052    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-dshm\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b04375e-536a-421a-8cf7-d1f57b07fe06-tls-certs\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-home\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-tmp-dir\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272182    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prrhq\" (UniqueName: \"kubernetes.io/projected/6b04375e-536a-421a-8cf7-d1f57b07fe06-kube-api-access-prrhq\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272229 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272226    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-model-cache\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272628 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272599    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-home\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272630    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-tmp-dir\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.272751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.272676    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-model-cache\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.274528 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.274489    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-dshm\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.274650 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.274634    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b04375e-536a-421a-8cf7-d1f57b07fe06-tls-certs\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.280787 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.280767    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prrhq\" (UniqueName: \"kubernetes.io/projected/6b04375e-536a-421a-8cf7-d1f57b07fe06-kube-api-access-prrhq\") pod \"rate-test-model-f01161-kserve-5d6cc6694c-cg5gk\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") " pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.429377 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.429288    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:20.564463 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:20.564438    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"]
Jul 07 02:50:20.566759 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:50:20.566721    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b04375e_536a_421a_8cf7_d1f57b07fe06.slice/crio-abf400399b23b0166b0fb358b6fa51c57fd1994d248e26704342a72eec5d703f WatchSource:0}: Error finding container abf400399b23b0166b0fb358b6fa51c57fd1994d248e26704342a72eec5d703f: Status 404 returned error can't find the container with id abf400399b23b0166b0fb358b6fa51c57fd1994d248e26704342a72eec5d703f
Jul 07 02:50:21.034787 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:21.034746    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk" event={"ID":"6b04375e-536a-421a-8cf7-d1f57b07fe06","Type":"ContainerStarted","Data":"01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744"}
Jul 07 02:50:21.034988 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:21.034794    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk" event={"ID":"6b04375e-536a-421a-8cf7-d1f57b07fe06","Type":"ContainerStarted","Data":"abf400399b23b0166b0fb358b6fa51c57fd1994d248e26704342a72eec5d703f"}
Jul 07 02:50:21.051943 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:21.051895    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk" podStartSLOduration=1.051881286 podStartE2EDuration="1.051881286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:50:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:50:21.051728271 +0000 UTC m=+2431.255149015" watchObservedRunningTime="2026-07-07 02:50:21.051881286 +0000 UTC m=+2431.255302029"
Jul 07 02:50:24.618887 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:24.618854    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:50:26.531422 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.531383    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq" podUID="3847a5de-66bb-4148-8705-2867f2363298" containerName="main" containerID="cri-o://9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c" gracePeriod=45
Jul 07 02:50:26.788409 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.788337    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:50:26.832312 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832280    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-tmp-dir\") pod \"3847a5de-66bb-4148-8705-2867f2363298\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") "
Jul 07 02:50:26.832469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832326    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-home\") pod \"3847a5de-66bb-4148-8705-2867f2363298\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") "
Jul 07 02:50:26.832469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832378    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-dshm\") pod \"3847a5de-66bb-4148-8705-2867f2363298\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") "
Jul 07 02:50:26.832469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c9r7\" (UniqueName: \"kubernetes.io/projected/3847a5de-66bb-4148-8705-2867f2363298-kube-api-access-9c9r7\") pod \"3847a5de-66bb-4148-8705-2867f2363298\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") "
Jul 07 02:50:26.832469 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832457    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3847a5de-66bb-4148-8705-2867f2363298-tls-certs\") pod \"3847a5de-66bb-4148-8705-2867f2363298\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") "
Jul 07 02:50:26.832704 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832481    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-model-cache\") pod \"3847a5de-66bb-4148-8705-2867f2363298\" (UID: \"3847a5de-66bb-4148-8705-2867f2363298\") "
Jul 07 02:50:26.832704 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832631    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3847a5de-66bb-4148-8705-2867f2363298" (UID: "3847a5de-66bb-4148-8705-2867f2363298"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:26.832890 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832859    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-model-cache" (OuterVolumeSpecName: "model-cache") pod "3847a5de-66bb-4148-8705-2867f2363298" (UID: "3847a5de-66bb-4148-8705-2867f2363298"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:26.832890 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.832646    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-home" (OuterVolumeSpecName: "home") pod "3847a5de-66bb-4148-8705-2867f2363298" (UID: "3847a5de-66bb-4148-8705-2867f2363298"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:26.834637 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.834612    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3847a5de-66bb-4148-8705-2867f2363298-kube-api-access-9c9r7" (OuterVolumeSpecName: "kube-api-access-9c9r7") pod "3847a5de-66bb-4148-8705-2867f2363298" (UID: "3847a5de-66bb-4148-8705-2867f2363298"). InnerVolumeSpecName "kube-api-access-9c9r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:50:26.834982 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.834950    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-dshm" (OuterVolumeSpecName: "dshm") pod "3847a5de-66bb-4148-8705-2867f2363298" (UID: "3847a5de-66bb-4148-8705-2867f2363298"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:26.835089 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.835031    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3847a5de-66bb-4148-8705-2867f2363298-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3847a5de-66bb-4148-8705-2867f2363298" (UID: "3847a5de-66bb-4148-8705-2867f2363298"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:50:26.933882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.933841    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:26.933882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.933874    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:26.933882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.933883    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:26.933882 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.933892    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c9r7\" (UniqueName: \"kubernetes.io/projected/3847a5de-66bb-4148-8705-2867f2363298-kube-api-access-9c9r7\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:26.934152 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.933901    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3847a5de-66bb-4148-8705-2867f2363298-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:26.934152 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:26.933909    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3847a5de-66bb-4148-8705-2867f2363298-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:27.066743 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.066659    2568 generic.go:356] "Generic (PLEG): container finished" podID="3847a5de-66bb-4148-8705-2867f2363298" containerID="9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c" exitCode=0
Jul 07 02:50:27.066743 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.066701    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"
Jul 07 02:50:27.066940 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.066735    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq" event={"ID":"3847a5de-66bb-4148-8705-2867f2363298","Type":"ContainerDied","Data":"9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c"}
Jul 07 02:50:27.066940 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.066780    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq" event={"ID":"3847a5de-66bb-4148-8705-2867f2363298","Type":"ContainerDied","Data":"ce6f5df8ab791b2cf444585cb989094d5bfedbf3e4939a3a9a8a00cfe2973f1f"}
Jul 07 02:50:27.066940 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.066797    2568 scope.go:117] "RemoveContainer" containerID="9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c"
Jul 07 02:50:27.076076 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.076060    2568 scope.go:117] "RemoveContainer" containerID="9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c"
Jul 07 02:50:27.076315 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:50:27.076293    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c\": container with ID starting with 9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c not found: ID does not exist" containerID="9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c"
Jul 07 02:50:27.076381 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.076327    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c"} err="failed to get container status \"9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c\": rpc error: code = NotFound desc = could not find container \"9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c\": container with ID starting with 9cfb0329f2c3fcce3c53a5fc4ebc126973933052cc2712bb226a4788a9538e8c not found: ID does not exist"
Jul 07 02:50:27.087953 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.087932    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"]
Jul 07 02:50:27.093839 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.093820    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/sub-test-model-f01161-kserve-6564c594fc-cmfhq"]
Jul 07 02:50:27.348734 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.348645    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq" podUID="bf4c5796-415c-4fe2-b491-c03961ff3775" containerName="main" containerID="cri-o://b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4" gracePeriod=45
Jul 07 02:50:27.594604 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.594560    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:50:27.641248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641150    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-home\") pod \"bf4c5796-415c-4fe2-b491-c03961ff3775\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") "
Jul 07 02:50:27.641248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641211    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4c5796-415c-4fe2-b491-c03961ff3775-tls-certs\") pod \"bf4c5796-415c-4fe2-b491-c03961ff3775\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") "
Jul 07 02:50:27.641248 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641243    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-model-cache\") pod \"bf4c5796-415c-4fe2-b491-c03961ff3775\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") "
Jul 07 02:50:27.641561 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641268    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-dshm\") pod \"bf4c5796-415c-4fe2-b491-c03961ff3775\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") "
Jul 07 02:50:27.641561 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641369    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4rjt\" (UniqueName: \"kubernetes.io/projected/bf4c5796-415c-4fe2-b491-c03961ff3775-kube-api-access-x4rjt\") pod \"bf4c5796-415c-4fe2-b491-c03961ff3775\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") "
Jul 07 02:50:27.641561 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641413    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-tmp-dir\") pod \"bf4c5796-415c-4fe2-b491-c03961ff3775\" (UID: \"bf4c5796-415c-4fe2-b491-c03961ff3775\") "
Jul 07 02:50:27.641561 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641438    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-home" (OuterVolumeSpecName: "home") pod "bf4c5796-415c-4fe2-b491-c03961ff3775" (UID: "bf4c5796-415c-4fe2-b491-c03961ff3775"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:27.641561 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641552    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf4c5796-415c-4fe2-b491-c03961ff3775" (UID: "bf4c5796-415c-4fe2-b491-c03961ff3775"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:27.641805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641735    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf4c5796-415c-4fe2-b491-c03961ff3775" (UID: "bf4c5796-415c-4fe2-b491-c03961ff3775"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:27.641805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641752    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:27.641805 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.641771    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:27.643400 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.643374    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-dshm" (OuterVolumeSpecName: "dshm") pod "bf4c5796-415c-4fe2-b491-c03961ff3775" (UID: "bf4c5796-415c-4fe2-b491-c03961ff3775"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:50:27.643864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.643833    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4c5796-415c-4fe2-b491-c03961ff3775-kube-api-access-x4rjt" (OuterVolumeSpecName: "kube-api-access-x4rjt") pod "bf4c5796-415c-4fe2-b491-c03961ff3775" (UID: "bf4c5796-415c-4fe2-b491-c03961ff3775"). InnerVolumeSpecName "kube-api-access-x4rjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:50:27.643864 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.643845    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf4c5796-415c-4fe2-b491-c03961ff3775-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf4c5796-415c-4fe2-b491-c03961ff3775" (UID: "bf4c5796-415c-4fe2-b491-c03961ff3775"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:50:27.743056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.743024    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4rjt\" (UniqueName: \"kubernetes.io/projected/bf4c5796-415c-4fe2-b491-c03961ff3775-kube-api-access-x4rjt\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:27.743056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.743053    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:27.743056 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.743064    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4c5796-415c-4fe2-b491-c03961ff3775-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:27.743297 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:27.743072    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4c5796-415c-4fe2-b491-c03961ff3775-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:50:28.073597 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.073565    2568 generic.go:356] "Generic (PLEG): container finished" podID="bf4c5796-415c-4fe2-b491-c03961ff3775" containerID="b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4" exitCode=0
Jul 07 02:50:28.073804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.073623    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"
Jul 07 02:50:28.073804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.073664    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq" event={"ID":"bf4c5796-415c-4fe2-b491-c03961ff3775","Type":"ContainerDied","Data":"b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4"}
Jul 07 02:50:28.073804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.073713    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq" event={"ID":"bf4c5796-415c-4fe2-b491-c03961ff3775","Type":"ContainerDied","Data":"d168613ab19160a9c68c303f4211f32331e614f501152128f5f767aeab0c766c"}
Jul 07 02:50:28.073804 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.073740    2568 scope.go:117] "RemoveContainer" containerID="b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4"
Jul 07 02:50:28.082901 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.082887    2568 scope.go:117] "RemoveContainer" containerID="b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4"
Jul 07 02:50:28.083132 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:50:28.083117    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4\": container with ID starting with b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4 not found: ID does not exist" containerID="b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4"
Jul 07 02:50:28.083189 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.083140    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4"} err="failed to get container status \"b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4\": rpc error: code = NotFound desc = could not find container \"b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4\": container with ID starting with b00e9b4a99da266dd5f620c971922c79f433e3278a5f129fcd77d382b8c81bf4 not found: ID does not exist"
Jul 07 02:50:28.095403 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.095380    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"]
Jul 07 02:50:28.101516 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.101481    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/sub-test-model-3dabf0-kserve-5f69877c87-w8nfq"]
Jul 07 02:50:28.434485 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.434405    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3847a5de-66bb-4148-8705-2867f2363298" path="/var/lib/kubelet/pods/3847a5de-66bb-4148-8705-2867f2363298/volumes"
Jul 07 02:50:28.434851 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:28.434837    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4c5796-415c-4fe2-b491-c03961ff3775" path="/var/lib/kubelet/pods/bf4c5796-415c-4fe2-b491-c03961ff3775/volumes"
Jul 07 02:50:30.433975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:30.433943    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:30.433975 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:30.433983    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:30.442684 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:30.442666    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:31.101340 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:31.101299    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:50:46.824467 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.824434    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"]
Jul 07 02:50:46.825049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.825034    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4c5796-415c-4fe2-b491-c03961ff3775" containerName="main"
Jul 07 02:50:46.825094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.825053    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4c5796-415c-4fe2-b491-c03961ff3775" containerName="main"
Jul 07 02:50:46.825094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.825072    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3847a5de-66bb-4148-8705-2867f2363298" containerName="main"
Jul 07 02:50:46.825094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.825081    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3847a5de-66bb-4148-8705-2867f2363298" containerName="main"
Jul 07 02:50:46.825204 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.825193    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3847a5de-66bb-4148-8705-2867f2363298" containerName="main"
Jul 07 02:50:46.825238 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.825210    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4c5796-415c-4fe2-b491-c03961ff3775" containerName="main"
Jul 07 02:50:46.829873 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.829856    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:46.832316 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.832276    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"rate-test-model-3dabf0-kserve-self-signed-certs\""
Jul 07 02:50:46.833330 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.833307    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"kube-root-ca.crt\""
Jul 07 02:50:46.833448 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.833426    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"openshift-service-ca.crt\""
Jul 07 02:50:46.833579 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.833429    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3dabf0\"/\"default-dockercfg-lcrs5\""
Jul 07 02:50:46.836906 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.836883    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"]
Jul 07 02:50:46.849879 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.849851    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:50:46.924772 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.924733    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-tmp-dir\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:46.924947 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.924786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-dshm\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:46.924947 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.924883    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td7n5\" (UniqueName: \"kubernetes.io/projected/cb84e094-638b-4fc0-a828-606b6741688a-kube-api-access-td7n5\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:46.925022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.924952    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb84e094-638b-4fc0-a828-606b6741688a-tls-certs\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:46.925022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.924971    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-home\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:46.925022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:46.924989    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-model-cache\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.025815 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.025779    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb84e094-638b-4fc0-a828-606b6741688a-tls-certs\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.025815 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.025812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-home\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026048 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.025829    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-model-cache\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026048 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.025846    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-tmp-dir\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026048 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.025873    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-dshm\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026048 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.025946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td7n5\" (UniqueName: \"kubernetes.io/projected/cb84e094-638b-4fc0-a828-606b6741688a-kube-api-access-td7n5\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026264 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.026182    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-model-cache\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026316 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.026262    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-home\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.026316 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.026291    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-tmp-dir\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.028152 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.028124    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-dshm\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.028307 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.028287    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb84e094-638b-4fc0-a828-606b6741688a-tls-certs\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.034772 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.034751    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-td7n5\" (UniqueName: \"kubernetes.io/projected/cb84e094-638b-4fc0-a828-606b6741688a-kube-api-access-td7n5\") pod \"rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") " pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.140058 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.139961    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:47.276979 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:47.276955    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"]
Jul 07 02:50:47.279568 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:50:47.279542    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb84e094_638b_4fc0_a828_606b6741688a.slice/crio-71cfdb652d9154d072f6065e20362bc450e7743e4c8504596407e2fefdaa4577 WatchSource:0}: Error finding container 71cfdb652d9154d072f6065e20362bc450e7743e4c8504596407e2fefdaa4577: Status 404 returned error can't find the container with id 71cfdb652d9154d072f6065e20362bc450e7743e4c8504596407e2fefdaa4577
Jul 07 02:50:48.175705 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:48.175662    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc" event={"ID":"cb84e094-638b-4fc0-a828-606b6741688a","Type":"ContainerStarted","Data":"4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44"}
Jul 07 02:50:48.175705 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:48.175707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc" event={"ID":"cb84e094-638b-4fc0-a828-606b6741688a","Type":"ContainerStarted","Data":"71cfdb652d9154d072f6065e20362bc450e7743e4c8504596407e2fefdaa4577"}
Jul 07 02:50:48.193655 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:48.193608    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc" podStartSLOduration=2.1935918660000002 podStartE2EDuration="2.193591866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:50:48.192235382 +0000 UTC m=+2458.395656127" watchObservedRunningTime="2026-07-07 02:50:48.193591866 +0000 UTC m=+2458.397012610"
Jul 07 02:50:50.726134 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:50.726097    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:50:57.140907 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:57.140852    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:57.140907 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:57.140922    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:57.153189 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:57.153166    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:50:57.229956 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:50:57.229928    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:51:09.216845 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:09.216813    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:09.312863 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:09.312828    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:12.725896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:12.725863    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:51:21.136190 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:21.136156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:21.236005 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:21.235968    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:26.719246 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:26.719212    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:51:29.044880 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:29.044845    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:29.148387 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:29.148353    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:33.614608 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:33.614566    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"]
Jul 07 02:51:33.622768 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:33.622729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:34.410783 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:34.410751    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"]
Jul 07 02:51:34.432307 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:34.432278    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:35.498034 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:35.497998    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:35.874967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:35.874887    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:36.041879 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:36.041840    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:36.142327 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:36.142244    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:36.299015 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:36.298986    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:39.225549 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:39.225517    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:51:40.953146 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:40.953113    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:51:48.634600 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.634519    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk" podUID="6b04375e-536a-421a-8cf7-d1f57b07fe06" containerName="main" containerID="cri-o://01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744" gracePeriod=45
Jul 07 02:51:48.884276 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.884252    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:51:48.922078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922004    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prrhq\" (UniqueName: \"kubernetes.io/projected/6b04375e-536a-421a-8cf7-d1f57b07fe06-kube-api-access-prrhq\") pod \"6b04375e-536a-421a-8cf7-d1f57b07fe06\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") "
Jul 07 02:51:48.922078 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922043    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b04375e-536a-421a-8cf7-d1f57b07fe06-tls-certs\") pod \"6b04375e-536a-421a-8cf7-d1f57b07fe06\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") "
Jul 07 02:51:48.922303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922094    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-model-cache\") pod \"6b04375e-536a-421a-8cf7-d1f57b07fe06\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") "
Jul 07 02:51:48.922303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922116    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-dshm\") pod \"6b04375e-536a-421a-8cf7-d1f57b07fe06\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") "
Jul 07 02:51:48.922303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922227    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-home\") pod \"6b04375e-536a-421a-8cf7-d1f57b07fe06\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") "
Jul 07 02:51:48.922303 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922279    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-tmp-dir\") pod \"6b04375e-536a-421a-8cf7-d1f57b07fe06\" (UID: \"6b04375e-536a-421a-8cf7-d1f57b07fe06\") "
Jul 07 02:51:48.922523 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922375    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b04375e-536a-421a-8cf7-d1f57b07fe06" (UID: "6b04375e-536a-421a-8cf7-d1f57b07fe06"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:48.922523 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922483    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-home" (OuterVolumeSpecName: "home") pod "6b04375e-536a-421a-8cf7-d1f57b07fe06" (UID: "6b04375e-536a-421a-8cf7-d1f57b07fe06"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:48.922650 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922627    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b04375e-536a-421a-8cf7-d1f57b07fe06" (UID: "6b04375e-536a-421a-8cf7-d1f57b07fe06"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:48.922814 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922796    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:48.922879 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922819    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:48.922879 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.922833    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:48.924188 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.924168    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b04375e-536a-421a-8cf7-d1f57b07fe06-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b04375e-536a-421a-8cf7-d1f57b07fe06" (UID: "6b04375e-536a-421a-8cf7-d1f57b07fe06"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:51:48.924188 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.924179    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-dshm" (OuterVolumeSpecName: "dshm") pod "6b04375e-536a-421a-8cf7-d1f57b07fe06" (UID: "6b04375e-536a-421a-8cf7-d1f57b07fe06"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:48.924512 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:48.924476    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b04375e-536a-421a-8cf7-d1f57b07fe06-kube-api-access-prrhq" (OuterVolumeSpecName: "kube-api-access-prrhq") pod "6b04375e-536a-421a-8cf7-d1f57b07fe06" (UID: "6b04375e-536a-421a-8cf7-d1f57b07fe06"). InnerVolumeSpecName "kube-api-access-prrhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:51:49.023468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.023435    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prrhq\" (UniqueName: \"kubernetes.io/projected/6b04375e-536a-421a-8cf7-d1f57b07fe06-kube-api-access-prrhq\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.023468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.023462    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b04375e-536a-421a-8cf7-d1f57b07fe06-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.023468 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.023471    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b04375e-536a-421a-8cf7-d1f57b07fe06-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.431346 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.431306    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc" podUID="cb84e094-638b-4fc0-a828-606b6741688a" containerName="main" containerID="cri-o://4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44" gracePeriod=45
Jul 07 02:51:49.481112 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.481079    2568 generic.go:356] "Generic (PLEG): container finished" podID="6b04375e-536a-421a-8cf7-d1f57b07fe06" containerID="01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744" exitCode=0
Jul 07 02:51:49.481236 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.481125    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"
Jul 07 02:51:49.481236 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.481165    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk" event={"ID":"6b04375e-536a-421a-8cf7-d1f57b07fe06","Type":"ContainerDied","Data":"01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744"}
Jul 07 02:51:49.481236 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.481208    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk" event={"ID":"6b04375e-536a-421a-8cf7-d1f57b07fe06","Type":"ContainerDied","Data":"abf400399b23b0166b0fb358b6fa51c57fd1994d248e26704342a72eec5d703f"}
Jul 07 02:51:49.481236 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.481226    2568 scope.go:117] "RemoveContainer" containerID="01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744"
Jul 07 02:51:49.516047 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.516021    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"]
Jul 07 02:51:49.521965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.521945    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f01161/rate-test-model-f01161-kserve-5d6cc6694c-cg5gk"]
Jul 07 02:51:49.542676 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.542654    2568 scope.go:117] "RemoveContainer" containerID="01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744"
Jul 07 02:51:49.543011 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:51:49.542993    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744\": container with ID starting with 01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744 not found: ID does not exist" containerID="01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744"
Jul 07 02:51:49.543094 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.543028    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744"} err="failed to get container status \"01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744\": rpc error: code = NotFound desc = could not find container \"01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744\": container with ID starting with 01e7bf28f819cc43b4e4838a0e5aac443cd94283b43511a85924a94d4d0bd744 not found: ID does not exist"
Jul 07 02:51:49.680672 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.680633    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:51:49.730179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730100    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-tmp-dir\") pod \"cb84e094-638b-4fc0-a828-606b6741688a\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") "
Jul 07 02:51:49.730179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730132    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td7n5\" (UniqueName: \"kubernetes.io/projected/cb84e094-638b-4fc0-a828-606b6741688a-kube-api-access-td7n5\") pod \"cb84e094-638b-4fc0-a828-606b6741688a\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") "
Jul 07 02:51:49.730398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730185    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-home\") pod \"cb84e094-638b-4fc0-a828-606b6741688a\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") "
Jul 07 02:51:49.730398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730204    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb84e094-638b-4fc0-a828-606b6741688a-tls-certs\") pod \"cb84e094-638b-4fc0-a828-606b6741688a\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") "
Jul 07 02:51:49.730398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730300    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-model-cache\") pod \"cb84e094-638b-4fc0-a828-606b6741688a\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") "
Jul 07 02:51:49.730398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730333    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-dshm\") pod \"cb84e094-638b-4fc0-a828-606b6741688a\" (UID: \"cb84e094-638b-4fc0-a828-606b6741688a\") "
Jul 07 02:51:49.730398 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730363    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb84e094-638b-4fc0-a828-606b6741688a" (UID: "cb84e094-638b-4fc0-a828-606b6741688a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:49.730711 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730549    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-home" (OuterVolumeSpecName: "home") pod "cb84e094-638b-4fc0-a828-606b6741688a" (UID: "cb84e094-638b-4fc0-a828-606b6741688a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:49.730711 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730576    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb84e094-638b-4fc0-a828-606b6741688a" (UID: "cb84e094-638b-4fc0-a828-606b6741688a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:49.730711 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730692    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-model-cache\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.730711 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730703    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-tmp-dir\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.730856 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.730717    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-home\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.732357 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.732333    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-dshm" (OuterVolumeSpecName: "dshm") pod "cb84e094-638b-4fc0-a828-606b6741688a" (UID: "cb84e094-638b-4fc0-a828-606b6741688a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:51:49.732465 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.732339    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb84e094-638b-4fc0-a828-606b6741688a-kube-api-access-td7n5" (OuterVolumeSpecName: "kube-api-access-td7n5") pod "cb84e094-638b-4fc0-a828-606b6741688a" (UID: "cb84e094-638b-4fc0-a828-606b6741688a"). InnerVolumeSpecName "kube-api-access-td7n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:51:49.732465 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.732358    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb84e094-638b-4fc0-a828-606b6741688a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb84e094-638b-4fc0-a828-606b6741688a" (UID: "cb84e094-638b-4fc0-a828-606b6741688a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:51:49.832142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.832112    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb84e094-638b-4fc0-a828-606b6741688a-dshm\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.832142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.832138    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td7n5\" (UniqueName: \"kubernetes.io/projected/cb84e094-638b-4fc0-a828-606b6741688a-kube-api-access-td7n5\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:49.832142 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:49.832150    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb84e094-638b-4fc0-a828-606b6741688a-tls-certs\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:51:50.433193 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.433153    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b04375e-536a-421a-8cf7-d1f57b07fe06" path="/var/lib/kubelet/pods/6b04375e-536a-421a-8cf7-d1f57b07fe06/volumes"
Jul 07 02:51:50.487865 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.487831    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb84e094-638b-4fc0-a828-606b6741688a" containerID="4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44" exitCode=0
Jul 07 02:51:50.488004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.487878    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"
Jul 07 02:51:50.488004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.487891    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc" event={"ID":"cb84e094-638b-4fc0-a828-606b6741688a","Type":"ContainerDied","Data":"4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44"}
Jul 07 02:51:50.488004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.487921    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc" event={"ID":"cb84e094-638b-4fc0-a828-606b6741688a","Type":"ContainerDied","Data":"71cfdb652d9154d072f6065e20362bc450e7743e4c8504596407e2fefdaa4577"}
Jul 07 02:51:50.488004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.487936    2568 scope.go:117] "RemoveContainer" containerID="4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44"
Jul 07 02:51:50.496349 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.496331    2568 scope.go:117] "RemoveContainer" containerID="4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44"
Jul 07 02:51:50.496619 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:51:50.496602    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44\": container with ID starting with 4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44 not found: ID does not exist" containerID="4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44"
Jul 07 02:51:50.496680 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.496627    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44"} err="failed to get container status \"4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44\": rpc error: code = NotFound desc = could not find container \"4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44\": container with ID starting with 4b8d65cbda75166bdad1c7fde95b9f37534919bc5f5e69c9a3434a637a150b44 not found: ID does not exist"
Jul 07 02:51:50.505623 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.505600    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"]
Jul 07 02:51:50.509125 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:50.509097    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3dabf0/rate-test-model-3dabf0-kserve-6f5df8fb8c-stvjc"]
Jul 07 02:51:52.433908 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:51:52.433872    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb84e094-638b-4fc0-a828-606b6741688a" path="/var/lib/kubelet/pods/cb84e094-638b-4fc0-a828-606b6741688a/volumes"
Jul 07 02:52:17.864731 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:17.864695    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:18.139479 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:18.139399    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:18.386372 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:18.386317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:18.558016 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:18.557979    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:18.996049 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:18.996018    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:19.112675 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.112625    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:19.345106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.345022    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"]
Jul 07 02:52:19.345329 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.345296    2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" podUID="1422ba11-1d45-4536-a833-093459bf686e" containerName="maas-api" containerID="cri-o://482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48" gracePeriod=30
Jul 07 02:52:19.374619 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.374594    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:19.595828 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.595770    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:52:19.604427 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.604401    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26j4m\" (UniqueName: \"kubernetes.io/projected/1422ba11-1d45-4536-a833-093459bf686e-kube-api-access-26j4m\") pod \"1422ba11-1d45-4536-a833-093459bf686e\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") "
Jul 07 02:52:19.604535 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.604474    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") pod \"1422ba11-1d45-4536-a833-093459bf686e\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") "
Jul 07 02:52:19.604588 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.604556    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1422ba11-1d45-4536-a833-093459bf686e-openshift-service-ca\") pod \"1422ba11-1d45-4536-a833-093459bf686e\" (UID: \"1422ba11-1d45-4536-a833-093459bf686e\") "
Jul 07 02:52:19.604884 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.604865    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1422ba11-1d45-4536-a833-093459bf686e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1422ba11-1d45-4536-a833-093459bf686e" (UID: "1422ba11-1d45-4536-a833-093459bf686e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:52:19.606613 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.606553    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1422ba11-1d45-4536-a833-093459bf686e-kube-api-access-26j4m" (OuterVolumeSpecName: "kube-api-access-26j4m") pod "1422ba11-1d45-4536-a833-093459bf686e" (UID: "1422ba11-1d45-4536-a833-093459bf686e"). InnerVolumeSpecName "kube-api-access-26j4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:52:19.606715 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.606679    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1422ba11-1d45-4536-a833-093459bf686e" (UID: "1422ba11-1d45-4536-a833-093459bf686e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:52:19.630475 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.630440    2568 generic.go:356] "Generic (PLEG): container finished" podID="1422ba11-1d45-4536-a833-093459bf686e" containerID="482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48" exitCode=0
Jul 07 02:52:19.630685 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.630482    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"
Jul 07 02:52:19.630685 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.630524    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" event={"ID":"1422ba11-1d45-4536-a833-093459bf686e","Type":"ContainerDied","Data":"482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48"}
Jul 07 02:52:19.630685 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.630590    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf" event={"ID":"1422ba11-1d45-4536-a833-093459bf686e","Type":"ContainerDied","Data":"2c82b137243527b691670ad4254d36a508efb7628e3ffc0424e6f3fdbe65e90f"}
Jul 07 02:52:19.630685 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.630615    2568 scope.go:117] "RemoveContainer" containerID="482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48"
Jul 07 02:52:19.639973 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.639948    2568 scope.go:117] "RemoveContainer" containerID="482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48"
Jul 07 02:52:19.640259 ip-10-0-143-152 kubenswrapper[2568]: E0707 02:52:19.640239    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48\": container with ID starting with 482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48 not found: ID does not exist" containerID="482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48"
Jul 07 02:52:19.640305 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.640268    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48"} err="failed to get container status \"482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48\": rpc error: code = NotFound desc = could not find container \"482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48\": container with ID starting with 482808875fee05a07452c033b17d74ea1f6ff530f50e0387f5987d16ac17bc48 not found: ID does not exist"
Jul 07 02:52:19.652903 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.652878    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"]
Jul 07 02:52:19.656896 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.656874    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3dabf0-76fb6fc75d-td7qf"]
Jul 07 02:52:19.705815 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.705790    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1422ba11-1d45-4536-a833-093459bf686e-openshift-service-ca\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:52:19.705815 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.705814    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26j4m\" (UniqueName: \"kubernetes.io/projected/1422ba11-1d45-4536-a833-093459bf686e-kube-api-access-26j4m\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:52:19.705965 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:19.705825    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1422ba11-1d45-4536-a833-093459bf686e-maas-api-tls\") on node \"ip-10-0-143-152.ec2.internal\" DevicePath \"\""
Jul 07 02:52:20.434213 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:20.434174    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1422ba11-1d45-4536-a833-093459bf686e" path="/var/lib/kubelet/pods/1422ba11-1d45-4536-a833-093459bf686e/volumes"
Jul 07 02:52:22.024733 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:22.024699    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d9lpl"]
Jul 07 02:52:24.942309 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:24.942274    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:25.007106 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:25.007074    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:25.226740 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:25.226664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:25.463066 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:25.463025    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:31.546032 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:31.546005    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-fgt75_fae8eb39-c5c4-49da-a675-7d7ab43e6dc2/copy-manifests/0.log"
Jul 07 02:52:31.553407 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:31.553383    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-fgt75_fae8eb39-c5c4-49da-a675-7d7ab43e6dc2/manager/0.log"
Jul 07 02:52:33.479022 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:33.478991    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lzfrf_b7b89f5f-bdf4-419c-b9f5-832751ca8de5/manager/0.log"
Jul 07 02:52:33.842562 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:33.842519    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2szcc_2c6fc69b-de8f-4447-8aa1-3834c84f4842/manager/0.log"
Jul 07 02:52:33.960627 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:33.960594    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-d9lpl_a72452b3-1a89-4a92-af84-b4f97071b2d8/limitador/0.log"
Jul 07 02:52:34.699074 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:34.699048    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d78774d-2q5wt_26a1c564-072d-42a9-95fe-d70d063b7561/kube-auth-proxy/0.log"
Jul 07 02:52:36.226328 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:36.226297    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-xwgvz_bd9b3005-cbb9-4057-9046-20a8588346a3/main/0.log"
Jul 07 02:52:39.867591 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.867554    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7l5m7/must-gather-fvjrl"]
Jul 07 02:52:39.867999 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.867967    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b04375e-536a-421a-8cf7-d1f57b07fe06" containerName="main"
Jul 07 02:52:39.867999 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.867982    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b04375e-536a-421a-8cf7-d1f57b07fe06" containerName="main"
Jul 07 02:52:39.867999 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.867999    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb84e094-638b-4fc0-a828-606b6741688a" containerName="main"
Jul 07 02:52:39.868099 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.868005    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb84e094-638b-4fc0-a828-606b6741688a" containerName="main"
Jul 07 02:52:39.868099 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.868025    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1422ba11-1d45-4536-a833-093459bf686e" containerName="maas-api"
Jul 07 02:52:39.868099 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.868030    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1422ba11-1d45-4536-a833-093459bf686e" containerName="maas-api"
Jul 07 02:52:39.868196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.868102    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1422ba11-1d45-4536-a833-093459bf686e" containerName="maas-api"
Jul 07 02:52:39.868196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.868113    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b04375e-536a-421a-8cf7-d1f57b07fe06" containerName="main"
Jul 07 02:52:39.868196 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.868119    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb84e094-638b-4fc0-a828-606b6741688a" containerName="main"
Jul 07 02:52:39.871344 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.871324    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:39.874179 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.874154    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7l5m7\"/\"openshift-service-ca.crt\""
Jul 07 02:52:39.874307 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.874154    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7l5m7\"/\"kube-root-ca.crt\""
Jul 07 02:52:39.874307 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.874224    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7l5m7\"/\"default-dockercfg-rbjgn\""
Jul 07 02:52:39.880697 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.880674    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l5m7/must-gather-fvjrl"]
Jul 07 02:52:39.887875 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.887852    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:39.988171 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.988141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log" limit=1024
Jul 07 02:52:39.999415 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.999385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4frn\" (UniqueName: \"kubernetes.io/projected/4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11-kube-api-access-b4frn\") pod \"must-gather-fvjrl\" (UID: \"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11\") " pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:39.999559 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:39.999482    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11-must-gather-output\") pod \"must-gather-fvjrl\" (UID: \"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11\") " pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:40.100572 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.100536    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4frn\" (UniqueName: \"kubernetes.io/projected/4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11-kube-api-access-b4frn\") pod \"must-gather-fvjrl\" (UID: \"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11\") " pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:40.100735 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.100622    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11-must-gather-output\") pod \"must-gather-fvjrl\" (UID: \"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11\") " pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:40.100972 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.100953    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11-must-gather-output\") pod \"must-gather-fvjrl\" (UID: \"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11\") " pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:40.109165 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.109140    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4frn\" (UniqueName: \"kubernetes.io/projected/4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11-kube-api-access-b4frn\") pod \"must-gather-fvjrl\" (UID: \"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11\") " pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:40.181694 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.181587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l5m7/must-gather-fvjrl"
Jul 07 02:52:40.324837 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.324769    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l5m7/must-gather-fvjrl"]
Jul 07 02:52:40.327636 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:52:40.327607    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a6ea7fd_f9e1_4bd0_8359_e078c7d86c11.slice/crio-b2e2db600419063c0e2ec9d16d7b282bdffd9c9ca11767c0410c1d84c46c53b2 WatchSource:0}: Error finding container b2e2db600419063c0e2ec9d16d7b282bdffd9c9ca11767c0410c1d84c46c53b2: Status 404 returned error can't find the container with id b2e2db600419063c0e2ec9d16d7b282bdffd9c9ca11767c0410c1d84c46c53b2
Jul 07 02:52:40.329550 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.329533    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:52:40.740737 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:40.740698    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/must-gather-fvjrl" event={"ID":"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11","Type":"ContainerStarted","Data":"b2e2db600419063c0e2ec9d16d7b282bdffd9c9ca11767c0410c1d84c46c53b2"}
Jul 07 02:52:41.749661 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:41.749624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/must-gather-fvjrl" event={"ID":"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11","Type":"ContainerStarted","Data":"1d58eb67568e7befd2385377fc98e339fc2c6903f2bb7b523bc10ba14fb83564"}
Jul 07 02:52:41.750073 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:41.749669    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/must-gather-fvjrl" event={"ID":"4a6ea7fd-f9e1-4bd0-8359-e078c7d86c11","Type":"ContainerStarted","Data":"5012d2026846cfc6f8c694d00498d7d8608d7fea6a435c463e21c28a47f4c91d"}
Jul 07 02:52:41.766831 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:41.766768    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7l5m7/must-gather-fvjrl" podStartSLOduration=1.8351526439999999 podStartE2EDuration="2.766745064s" totalImagesPullingTime="931.59242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:52:41.764859589 +0000 UTC m=+2571.968280337" watchObservedRunningTime="2026-07-07 02:52:41.766745064 +0000 UTC m=+2571.970165808"
Jul 07 02:52:42.949950 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:42.949923    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q9bb5_a4358822-184a-4fa7-8d72-27b1af7c6d75/global-pull-secret-syncer/0.log"
Jul 07 02:52:43.000751 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:43.000717    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-57vj2_bec11113-359b-4151-bd2b-cf5937c9879e/konnectivity-agent/0.log"
Jul 07 02:52:43.131759 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:43.131732    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-152.ec2.internal_a5a6799705e4948b663271414d9e6b73/haproxy/0.log"
Jul 07 02:52:47.779915 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:47.779882    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lzfrf_b7b89f5f-bdf4-419c-b9f5-832751ca8de5/manager/0.log"
Jul 07 02:52:48.057750 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:48.057623    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2szcc_2c6fc69b-de8f-4447-8aa1-3834c84f4842/manager/0.log"
Jul 07 02:52:48.078630 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:48.078589    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-d9lpl_a72452b3-1a89-4a92-af84-b4f97071b2d8/limitador/0.log"
Jul 07 02:52:49.721313 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.721278    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/alertmanager/0.log"
Jul 07 02:52:49.749177 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.749151    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/config-reloader/0.log"
Jul 07 02:52:49.772772 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.772746    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/kube-rbac-proxy-web/0.log"
Jul 07 02:52:49.795166 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.795129    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/kube-rbac-proxy/0.log"
Jul 07 02:52:49.820212 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.820184    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/kube-rbac-proxy-metric/0.log"
Jul 07 02:52:49.841256 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.841223    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/prom-label-proxy/0.log"
Jul 07 02:52:49.864463 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:49.864419    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ab5ff5b1-2973-40bd-bf04-c82cb710ae9f/init-config-reloader/0.log"
Jul 07 02:52:50.062629 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.062597    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5cbfc84df-p225k_5aa3034f-8e93-4e74-907c-be6fa9bc1085/metrics-server/0.log"
Jul 07 02:52:50.097810 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.097776    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-89j5z_4dfd0163-83d3-415b-a97b-ea13e64a2afb/monitoring-plugin/0.log"
Jul 07 02:52:50.440967 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.440907    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kw7tf_e8fbc0dc-24ac-4580-8c92-52773eb35827/node-exporter/0.log"
Jul 07 02:52:50.479898 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.479849    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kw7tf_e8fbc0dc-24ac-4580-8c92-52773eb35827/kube-rbac-proxy/0.log"
Jul 07 02:52:50.509641 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.509595    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kw7tf_e8fbc0dc-24ac-4580-8c92-52773eb35827/init-textfile/0.log"
Jul 07 02:52:50.683689 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.683643    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/prometheus/0.log"
Jul 07 02:52:50.711005 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.710974    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/config-reloader/0.log"
Jul 07 02:52:50.747890 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.747859    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/thanos-sidecar/0.log"
Jul 07 02:52:50.777761 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.777690    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/kube-rbac-proxy-web/0.log"
Jul 07 02:52:50.811009 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.810970    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/kube-rbac-proxy/0.log"
Jul 07 02:52:50.832220 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.832186    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/kube-rbac-proxy-thanos/0.log"
Jul 07 02:52:50.856809 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:50.856779    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_342653ff-72c9-4626-b264-00bd3078e7b6/init-config-reloader/0.log"
Jul 07 02:52:51.063796 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:51.063762    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d79695c-592zr_6c1e295e-29cc-4033-be75-609fffee5be9/thanos-query/0.log"
Jul 07 02:52:51.084822 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:51.084786    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d79695c-592zr_6c1e295e-29cc-4033-be75-609fffee5be9/kube-rbac-proxy-web/0.log"
Jul 07 02:52:51.106295 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:51.106263    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d79695c-592zr_6c1e295e-29cc-4033-be75-609fffee5be9/kube-rbac-proxy/0.log"
Jul 07 02:52:51.126300 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:51.126272    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d79695c-592zr_6c1e295e-29cc-4033-be75-609fffee5be9/prom-label-proxy/0.log"
Jul 07 02:52:51.146406 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:51.146377    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d79695c-592zr_6c1e295e-29cc-4033-be75-609fffee5be9/kube-rbac-proxy-rules/0.log"
Jul 07 02:52:51.169861 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:51.169829    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d79695c-592zr_6c1e295e-29cc-4033-be75-609fffee5be9/kube-rbac-proxy-metrics/0.log"
Jul 07 02:52:53.017254 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.017221    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/1.log"
Jul 07 02:52:53.023237 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.023206    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-sdhnx_45f4fec7-722a-4b66-98c1-185c65ad2816/console-operator/2.log"
Jul 07 02:52:53.561526 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.561484    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d8f4bf44b-wglvc_3a60b967-f9ad-4620-91f4-4759ebd899bd/console/0.log"
Jul 07 02:52:53.599848 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.599807    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-vk9xl_1c8724da-6d8a-40d9-b1f1-821764073753/download-server/0.log"
Jul 07 02:52:53.713847 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.713799    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"]
Jul 07 02:52:53.720522 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.720467    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.731101 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.731069    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"]
Jul 07 02:52:53.770082 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.770042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-sys\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.770276 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.770139    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpx7n\" (UniqueName: \"kubernetes.io/projected/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-kube-api-access-bpx7n\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.770276 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.770210    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-lib-modules\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.770399 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.770304    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-proc\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.770399 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.770354    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-podres\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.871159 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-proc\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.871432 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871413    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-podres\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.871620 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871603    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-sys\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.871775 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpx7n\" (UniqueName: \"kubernetes.io/projected/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-kube-api-access-bpx7n\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.872277 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.872253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-lib-modules\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.872385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871678    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-sys\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.872385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871296    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-proc\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.872385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.871627    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-podres\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.872571 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.872517    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-lib-modules\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:53.882208 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:53.882181    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpx7n\" (UniqueName: \"kubernetes.io/projected/9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559-kube-api-access-bpx7n\") pod \"perf-node-gather-daemonset-6stqz\" (UID: \"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:54.042194 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:54.042150    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:54.226037 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:54.225919    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"]
Jul 07 02:52:54.232438 ip-10-0-143-152 kubenswrapper[2568]: W0707 02:52:54.232395    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9b6f6a9b_e5e6_4bac_bfb7_8009dc73e559.slice/crio-b6b42b9cec4c8ac0c8ae8a2c75c4e29278d3ce51b306074acd130bb6e41c9a2f WatchSource:0}: Error finding container b6b42b9cec4c8ac0c8ae8a2c75c4e29278d3ce51b306074acd130bb6e41c9a2f: Status 404 returned error can't find the container with id b6b42b9cec4c8ac0c8ae8a2c75c4e29278d3ce51b306074acd130bb6e41c9a2f
Jul 07 02:52:54.851889 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:54.850910    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz" event={"ID":"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559","Type":"ContainerStarted","Data":"82ee1568fa33ad4626c076d48b1f97933c03e403393dca601de2c97aa5a03b09"}
Jul 07 02:52:54.851889 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:54.850971    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz" event={"ID":"9b6f6a9b-e5e6-4bac-bfb7-8009dc73e559","Type":"ContainerStarted","Data":"b6b42b9cec4c8ac0c8ae8a2c75c4e29278d3ce51b306074acd130bb6e41c9a2f"}
Jul 07 02:52:54.851889 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:54.851825    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:52:54.868593 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:54.868522    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz" podStartSLOduration=1.8684823069999998 podStartE2EDuration="1.868482307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:52:54.867139932 +0000 UTC m=+2585.070560679" watchObservedRunningTime="2026-07-07 02:52:54.868482307 +0000 UTC m=+2585.071903051"
Jul 07 02:52:55.435397 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:55.435363    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9zcvr_a5a8d96c-2c84-479e-a76c-de376aca0782/dns/0.log"
Jul 07 02:52:55.454612 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:55.454586    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9zcvr_a5a8d96c-2c84-479e-a76c-de376aca0782/kube-rbac-proxy/0.log"
Jul 07 02:52:55.503290 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:55.503263    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2wnp_b6144ddf-5525-45aa-afb7-ef69d27cab03/dns-node-resolver/0.log"
Jul 07 02:52:56.100389 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:56.100355    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r7s9h_d500a857-2643-4108-80af-8da289f0f3c7/node-ca/0.log"
Jul 07 02:52:57.171020 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:57.170987    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d78774d-2q5wt_26a1c564-072d-42a9-95fe-d70d063b7561/kube-auth-proxy/0.log"
Jul 07 02:52:58.056133 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:58.056103    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xt4rs_a387479e-f772-49bc-b157-c146c9358d3e/serve-healthcheck-canary/0.log"
Jul 07 02:52:58.530983 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:58.530942    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7prkb_a77aff70-34c1-4ab4-a4be-901ecebc8a7c/insights-operator/0.log"
Jul 07 02:52:58.531624 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:58.531605    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7prkb_a77aff70-34c1-4ab4-a4be-901ecebc8a7c/insights-operator/1.log"
Jul 07 02:52:58.622961 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:58.622930    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nh85c_d60bc751-eec6-4ff7-9a52-aad22cfeb3ce/kube-rbac-proxy/0.log"
Jul 07 02:52:58.643928 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:58.643901    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nh85c_d60bc751-eec6-4ff7-9a52-aad22cfeb3ce/exporter/0.log"
Jul 07 02:52:58.666127 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:52:58.666093    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nh85c_d60bc751-eec6-4ff7-9a52-aad22cfeb3ce/extractor/0.log"
Jul 07 02:53:01.393222 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:01.393183    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-fgt75_fae8eb39-c5c4-49da-a675-7d7ab43e6dc2/manager/0.log"
Jul 07 02:53:01.414890 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:01.414865    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-fgt75_fae8eb39-c5c4-49da-a675-7d7ab43e6dc2/copy-manifests/0.log"
Jul 07 02:53:01.873428 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:01.873400    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-6stqz"
Jul 07 02:53:02.832108 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:02.832071    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bd684c58c-fsx85_fab27ad1-fa70-4ee0-aa8a-7dd561b620f2/manager/0.log"
Jul 07 02:53:03.609530 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:03.609489    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d78774d-2q5wt_26a1c564-072d-42a9-95fe-d70d063b7561/kube-auth-proxy/0.log"
Jul 07 02:53:05.274385 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:05.274360    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d78774d-2q5wt_26a1c564-072d-42a9-95fe-d70d063b7561/kube-auth-proxy/0.log"
Jul 07 02:53:09.997574 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:09.997542    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/kube-multus-additional-cni-plugins/0.log"
Jul 07 02:53:10.017707 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.017679    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/egress-router-binary-copy/0.log"
Jul 07 02:53:10.038091 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.038064    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/cni-plugins/0.log"
Jul 07 02:53:10.058575 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.058545    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/bond-cni-plugin/0.log"
Jul 07 02:53:10.079438 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.079414    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/routeoverride-cni/0.log"
Jul 07 02:53:10.101004 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.100973    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/whereabouts-cni-bincopy/0.log"
Jul 07 02:53:10.120857 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.120834    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrrzv_ec8c127f-e39b-4e01-a740-8e5b4a5bf0c7/whereabouts-cni/0.log"
Jul 07 02:53:10.189862 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.189830    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mrpsn_5eedf342-ae5c-45ce-bc64-474b56286922/kube-multus/0.log"
Jul 07 02:53:10.307343 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.307236    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mzcc5_478e76b1-c8b0-466d-aa7e-5dc477d27b41/network-metrics-daemon/0.log"
Jul 07 02:53:10.328998 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:10.328969    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mzcc5_478e76b1-c8b0-466d-aa7e-5dc477d27b41/kube-rbac-proxy/0.log"
Jul 07 02:53:11.603307 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.603221    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/ovn-controller/0.log"
Jul 07 02:53:11.619171 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.619145    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/ovn-acl-logging/0.log"
Jul 07 02:53:11.646885 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.646856    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/kube-rbac-proxy-node/0.log"
Jul 07 02:53:11.669386 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.669340    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 02:53:11.686031 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.686002    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/northd/0.log"
Jul 07 02:53:11.707484 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.707454    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/nbdb/0.log"
Jul 07 02:53:11.728728 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.728677    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/sbdb/0.log"
Jul 07 02:53:11.787694 ip-10-0-143-152 kubenswrapper[2568]: I0707 02:53:11.787665    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9t55_ba250c8f-a604-4c9c-9f8b-ad4d69dd4b5e/ovnkube-controller/0.log"