Jul 19 14:11:08.811556 ip-10-0-143-90 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 19 14:11:08.811565 ip-10-0-143-90 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 19 14:11:08.811572 ip-10-0-143-90 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 19 14:11:08.811796 ip-10-0-143-90 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 19 14:11:18.921410 ip-10-0-143-90 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 19 14:11:18.921426 ip-10-0-143-90 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 90285f6e21a04c05b65b4bdd464c3191 --
Jul 19 14:13:01.596807 ip-10-0-143-90 systemd[1]: Starting Kubernetes Kubelet...
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.936745    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945330    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945342    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945347    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945350    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945353    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945356    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945359    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:13:02.043799 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945362    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:13:02.002305 ip-10-0-143-90 systemd[1]: Started Kubernetes Kubelet.
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945365    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945367    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945370    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945373    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945375    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945378    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945381    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945383    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945386    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945389    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945391    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945394    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945397    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945399    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945402    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945404    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945407    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945410    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945412    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:13:02.046347 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945415    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945418    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945420    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945423    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945428    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945432    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945435    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945438    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945440    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945444    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945447    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945451    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945454    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945456    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945459    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945462    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945465    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945468    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945470    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945473    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:13:02.053126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945475    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945479    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945482    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945485    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945487    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945490    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945493    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945495    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945498    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945500    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945502    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945505    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945507    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945511    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945513    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945516    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945532    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945536    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945538    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:13:02.057837 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945541    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945544    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945546    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945549    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945552    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945556    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945559    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945561    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945564    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945566    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945569    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945571    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945574    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945577    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945579    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945581    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945585    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945588    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945590    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945593    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:13:02.064635 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945595    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945598    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945600    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945602    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945605    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945607    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945610    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945612    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945615    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945617    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945620    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945622    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.945626    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946283    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946290    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946292    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946295    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946298    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946301    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946305    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:13:02.065863 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946309    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946312    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946315    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946318    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946321    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946324    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946327    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946329    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946332    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946335    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946338    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946341    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946343    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946346    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946348    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946351    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946354    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946356    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946359    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946362    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:13:02.068954 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946364    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946367    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946369    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946372    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946374    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946377    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946381    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946384    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946387    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946391    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946394    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946398    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946401    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946404    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946407    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946411    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946413    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946416    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946418    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:13:02.070113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946421    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946424    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946426    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946429    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946431    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946434    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946436    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946439    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946441    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946444    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946447    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946449    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946452    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946454    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946457    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946459    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946462    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946464    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946467    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946469    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:13:02.073949 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946472    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946475    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946478    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946480    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946483    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946487    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946489    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946492    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946494    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946497    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946499    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946502    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946504    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946507    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946509    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946512    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946514    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946518    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946538    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946547    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:13:02.075108 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946551    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946555    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946559    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946562    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946565    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946567    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946570    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946572    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946575    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946577    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946580    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946582    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946585    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946989    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946994    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.946998    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947001    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947004    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947007    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947010    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:13:02.078996 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947013    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947015    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947018    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947020    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947023    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947025    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947028    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947031    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947033    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947036    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947039    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947041    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947044    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947047    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947050    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947052    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947055    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947057    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947059    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947062    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:13:02.080042 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947066    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947069    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947071    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947075    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947078    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947081    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947084    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947086    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947089    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947091    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947094    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947096    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947099    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947101    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947103    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947106    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947109    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947111    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947113    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:13:02.083158 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947116    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947118    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947120    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947123    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947125    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947128    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947131    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947133    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947136    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947139    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947142    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947146    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947149    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947153    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947156    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947158    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947161    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947164    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947167    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:13:02.083912 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947170    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947172    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947175    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947178    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947181    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947183    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947185    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947188    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947190    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947193    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947195    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947198    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947200    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947203    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947206    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947208    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947210    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947214    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947217    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947220    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:13:02.086596 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947222    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947225    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947227    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947229    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947232    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947234    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947239    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947242    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947246    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947249    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947252    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947254    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947257    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947259    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947325    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947332    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947340    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947344    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947348    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947351    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947355    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 19 14:13:02.087334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947359    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947362    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947365    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947368    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947372    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947375    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947377    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947380    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947383    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947387    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947390    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947394    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947397    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947399    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947402    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947406    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947409    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947413    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947416    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947419    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947422    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947425    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947428    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947431    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 19 14:13:02.091202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947434    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947438    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947440    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947444    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947447    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947450    2574 flags.go:64] FLAG: --enable-server="true"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947453    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947457    2574 flags.go:64] FLAG: --event-burst="100"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947460    2574 flags.go:64] FLAG: --event-qps="50"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947463    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947466    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947469    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947473    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947475    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947478    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947481    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947484    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947487    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947496    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947499    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947502    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947504    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947507    2574 flags.go:64] FLAG: --feature-gates=""
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947510    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947513    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 19 14:13:02.092328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947532    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947535    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947539    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947542    2574 flags.go:64] FLAG: --help="false"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947544    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947547    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947550    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947553    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947556    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947559    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947564    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947567    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947570    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947573    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947575    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947578    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947581    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947584    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947587    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947590    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947592    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947595    2574 flags.go:64] FLAG: --lock-file=""
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947598    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947601    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 19 14:13:02.166725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947603    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947608    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947619    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947622    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947625    2574 flags.go:64] FLAG: --logging-format="text"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947628    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947631    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947634    2574 flags.go:64] FLAG: --manifest-url=""
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947637    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947642    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947644    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947648    2574 flags.go:64] FLAG: --max-pods="110"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947651    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947654    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947656    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947659    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947662    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947665    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947669    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947676    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947679    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947682    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947685    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 19 14:13:02.167918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947688    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947694    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947696    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947699    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947702    2574 flags.go:64] FLAG: --port="10250"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947705    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947708    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00df3c398eff1e530"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947711    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947714    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947716    2574 flags.go:64] FLAG: --register-node="true"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947719    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947722    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947726    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947728    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947731    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947735    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947738    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947741    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947745    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947748    2574 flags.go:64] FLAG: --runonce="false"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947751    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947754    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947757    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947759    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947762    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947765    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 19 14:13:02.168946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947768    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947773    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947775    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947778    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947781    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947784    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947786    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947789    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947792    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947797    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947800    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947802    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947806    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947809    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947812    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947815    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947817    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947820    2574 flags.go:64] FLAG: --v="2"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947824    2574 flags.go:64] FLAG: --version="false"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947831    2574 flags.go:64] FLAG: --vmodule=""
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947834    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.947838    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947941    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947945    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947950    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:13:02.170009 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947953    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947956    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947958    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947961    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947964    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947966    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947969    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947971    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947976    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947978    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947981    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947983    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947985    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.947988    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948004    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948007    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948010    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948013    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948016    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948018    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:13:02.170858 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948021    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948024    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948026    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948029    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948031    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948034    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948036    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948039    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948041    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948044    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948047    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948051    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948053    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948056    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948058    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948061    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948063    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948066    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948069    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:13:02.171513 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948072    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948077    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948080    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948083    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948086    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948088    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948091    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948094    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948096    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948099    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948101    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948103    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948106    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948108    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948111    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948113    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948116    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948119    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948121    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948123    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:13:02.173060 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948126    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948129    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948131    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948134    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948137    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948140    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948143    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948145    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948147    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948150    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948152    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948155    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948157    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948162    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948164    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948166    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948169    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948171    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948174    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948176    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:13:02.173693 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948179    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948181    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948184    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948186    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948188    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948191    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948194    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948196    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948200    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948203    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948206    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948208    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948211    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948214    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948218    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948222    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:13:02.175798 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.948226    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.948810    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955045    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955059    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955076    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955085    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955134    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955138    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955142    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955145    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955148    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955151    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955153    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955156    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955158    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:13:02.176248 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955161    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955164    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955166    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955169    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955171    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955174    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955176    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955179    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955182    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955186    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955191    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955194    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955196    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955199    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955201    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955209    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955212    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955216    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955219    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955221    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:13:02.258771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955224    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955227    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955229    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955231    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955234    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955236    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955239    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955243    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955246    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955249    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955252    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955254    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955257    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955260    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955262    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955265    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955268    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955271    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955274    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955278    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:13:02.281173 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955281    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955284    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955287    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955289    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955292    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955294    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955297    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955299    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955303    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955305    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955308    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955310    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955312    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955315    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955317    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955320    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955322    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955325    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955327    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955330    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:13:02.289103 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955332    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955335    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955337    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955340    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955342    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955345    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955347    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955349    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955352    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955355    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955358    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955363    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955365    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955368    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955374    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955377    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955380    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955383    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955385    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:13:02.290222 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955388    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955391    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955393    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955396    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955399    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955401    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955404    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955406    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955409    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955411    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955414    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955418    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955534    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955538    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:13:02.291016 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955541    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955544    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955547    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955549    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955552    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955554    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955557    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955559    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955562    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955564    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955568    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955572    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955576    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955579    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955583    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955586    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955588    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955591    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955594    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:13:02.292497 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955596    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955599    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955602    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955605    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955607    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955610    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955612    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955614    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955617    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955621    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955624    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955627    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955630    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955632    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955635    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955638    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955641    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955644    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955646    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:13:02.293258 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955649    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955651    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955654    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955656    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955660    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955662    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955665    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955667    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955671    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955674    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955677    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955680    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955682    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955684    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955687    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955689    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955692    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955695    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955698    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955700    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:13:02.294395 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955702    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955705    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955707    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955710    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955713    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955715    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955718    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955720    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955723    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955725    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955727    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955730    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955732    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955735    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955737    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955740    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955744    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955746    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955749    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955751    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:13:02.295114 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955756    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955759    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955761    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955764    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955766    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955769    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955771    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955774    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955776    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955780    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955782    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955785    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955787    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955789    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955792    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955794    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955797    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955799    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:13:02.296252 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:01.955802    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955807    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.955888    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.957919    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.959172    2574 server.go:1019] "Starting client certificate rotation"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.959271    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.959901    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.961374    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.963056    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.972073    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.974152    2574 log.go:25] "Validated CRI v1 image API"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.974998    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 19 14:13:02.298114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.977007    2574 fs.go:135] Filesystem UUIDs: map[008b644c-b5ed-4fee-917f-b1ec99d07a08:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b71fac74-d060-4b94-8c30-619fec4f954f:/dev/nvme0n1p4]
Jul 19 14:13:02.298460 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.977020    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.982377    2574 manager.go:216] Machine: {Timestamp:2026-07-19 14:13:01.98114646 +0000 UTC m=+0.283977466 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203824 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b60b2ae8066fb814d53adc2f07610 SystemUUID:ec2b60b2-ae80-66fb-814d-53adc2f07610 BootID:90285f6e-21a0-4c05-b65b-4bdd464c3191 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:c1:27:07:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:c1:27:07:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:98:2e:83:92:a7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.982466    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.982553    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.985193    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.985221    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-90.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.985910    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.985919    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 19 14:13:02.298556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.985926    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.986560    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.988073    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.989638    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.991988    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992102    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992129    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992140    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992154    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992163    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992170    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992501    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.992535    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.994661    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.996656    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.996682    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997822    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997836    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997846    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997853    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997861    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997866    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997874    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997879    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997885    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997890    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997898    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.997903    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.998571    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:01.998578    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.001393    2574 server.go:1268] "Started kubelet"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.001508    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.001517    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.001662    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.003365    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.005004    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.006333    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.006361    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-90.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.006474    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 19 14:13:02.298795 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.007823    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.011690    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.011783    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.011976    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012299    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012301    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012320    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012439    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012448    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012513    2574 factory.go:55] Registering systemd factory
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012557    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.011895    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-90.ec2.internal.18c3b65b9f65ab24  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-90.ec2.internal,UID:ip-10-0-143-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-90.ec2.internal,},FirstTimestamp:2026-07-19 14:13:02.001367844 +0000 UTC m=+0.304198852,LastTimestamp:2026-07-19 14:13:02.001367844 +0000 UTC m=+0.304198852,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-90.ec2.internal,}"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012733    2574 factory.go:153] Registering CRI-O factory
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012743    2574 factory.go:223] Registration of the crio container factory successfully
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012788    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012812    2574 factory.go:103] Registering Raw factory
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.012825    2574 manager.go:1196] Started watching for new ooms in manager
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.013757    2574 manager.go:349] Starting recovery of all containers
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.024821    2574 manager.go:354] Recovery completed
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.026971    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.027001    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.028323    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.030491    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033196    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033227    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033238    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033821    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033830    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033834    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.299802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.033843    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.035367    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-90.ec2.internal.18c3b65ba14b8fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-90.ec2.internal,UID:ip-10-0-143-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-90.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-90.ec2.internal,},FirstTimestamp:2026-07-19 14:13:02.033211373 +0000 UTC m=+0.336042382,LastTimestamp:2026-07-19 14:13:02.033211373 +0000 UTC m=+0.336042382,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-90.ec2.internal,}"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.035984    2574 policy_none.go:49] "None policy: Start"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.035995    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.036005    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.036892    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.036913    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.036922    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.038057    2574 policy_none.go:47] "Start"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.104953    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dcd2v"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.112033    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.112184    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dcd2v"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.163116    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-90.ec2.internal.18c3b65ba14be296  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-90.ec2.internal,UID:ip-10-0-143-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-90.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-90.ec2.internal,},FirstTimestamp:2026-07-19 14:13:02.033232534 +0000 UTC m=+0.336063540,LastTimestamp:2026-07-19 14:13:02.033232534 +0000 UTC m=+0.336063540,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-90.ec2.internal,}"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206019    2574 manager.go:337] "Starting Device Plugin manager"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.206161    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206171    2574 server.go:87] "Starting device plugin registration server"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206352    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206361    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206661    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206723    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.206729    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.207659    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.207707    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.300530 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.232991    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-90.ec2.internal\" not found" node="ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.307125 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.307102    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.308132 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.308107    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.308236 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.308147    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.308236 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.308159    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.308236 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.308186    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.316137 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.316120    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.316215 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.316142    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-90.ec2.internal\": node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.329468 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.329449    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.390476 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.390451    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 19 14:13:02.390657 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.390492    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 19 14:13:02.390657 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.390511    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 19 14:13:02.390657 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.390570    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 19 14:13:02.392775 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.392757    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:02.430233 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.430208    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.491559 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.491494    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"]
Jul 19 14:13:02.491639 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.491605    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.493573 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.493558    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.493647 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.493589    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.493647 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.493603    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.496027 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496015    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.496157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496143    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.496193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496172    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.496830 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496816    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.496830 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496825    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.496940 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496843    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.496940 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496846    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.496940 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496869    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.496940 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.496857    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.499199 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.499183    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.499273 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.499214    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:13:02.501463 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.501449    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:13:02.501561 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.501475    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:13:02.501561 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.501489    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:13:02.517721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.517706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eabed4ba2b3d6a2cc98e02bb7bec66ad-config\") pod \"kube-apiserver-proxy-ip-10-0-143-90.ec2.internal\" (UID: \"eabed4ba2b3d6a2cc98e02bb7bec66ad\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.517787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.517728    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7dc26ae2909348f6daa2c1d449c4ac5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal\" (UID: \"a7dc26ae2909348f6daa2c1d449c4ac5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.517787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.517746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7dc26ae2909348f6daa2c1d449c4ac5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal\" (UID: \"a7dc26ae2909348f6daa2c1d449c4ac5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.528104 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.528089    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-90.ec2.internal\" not found" node="ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.530343 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.530325    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.532345 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.532330    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-90.ec2.internal\" not found" node="ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.617943 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.617908    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7dc26ae2909348f6daa2c1d449c4ac5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal\" (UID: \"a7dc26ae2909348f6daa2c1d449c4ac5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.618065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.617947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7dc26ae2909348f6daa2c1d449c4ac5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal\" (UID: \"a7dc26ae2909348f6daa2c1d449c4ac5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.618065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.617992    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7dc26ae2909348f6daa2c1d449c4ac5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal\" (UID: \"a7dc26ae2909348f6daa2c1d449c4ac5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.618065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.617999    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7dc26ae2909348f6daa2c1d449c4ac5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal\" (UID: \"a7dc26ae2909348f6daa2c1d449c4ac5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.618065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.618051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eabed4ba2b3d6a2cc98e02bb7bec66ad-config\") pod \"kube-apiserver-proxy-ip-10-0-143-90.ec2.internal\" (UID: \"eabed4ba2b3d6a2cc98e02bb7bec66ad\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.618222 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.618102    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eabed4ba2b3d6a2cc98e02bb7bec66ad-config\") pod \"kube-apiserver-proxy-ip-10-0-143-90.ec2.internal\" (UID: \"eabed4ba2b3d6a2cc98e02bb7bec66ad\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.631332 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.631308    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.732315 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.732276    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.831699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.831628    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.832696 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:02.832681    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-90.ec2.internal\" not found"
Jul 19 14:13:02.835176 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.835158    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:02.917331 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.917312    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:02.926649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.926631    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:02.958892 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.958871    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 19 14:13:02.959002 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.958977    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:13:02.959095 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.959011    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:13:02.959095 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.959022    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:13:02.993325 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:02.993303    2574 apiserver.go:52] "Watching apiserver"
Jul 19 14:13:03.010722 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.010698    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 19 14:13:03.011014 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.010991    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc","openshift-cluster-node-tuning-operator/tuned-4j4bz","openshift-multus/multus-additional-cni-plugins-ssfg4","openshift-multus/multus-mkk5t","openshift-multus/network-metrics-daemon-6kfzr","openshift-network-diagnostics/network-check-target-cdvtc","openshift-network-operator/iptables-alerter-rsxw5","openshift-ovn-kubernetes/ovnkube-node-h4r72","kube-system/global-pull-secret-syncer-x7d24","kube-system/konnectivity-agent-2mfrj","openshift-dns/node-resolver-597hr","openshift-image-registry/node-ca-kxf6j"]
Jul 19 14:13:03.012496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.012475    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:03.012587 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.012511    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 19 14:13:03.013658 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.013640    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.015953 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.015937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.016189 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.016060    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 19 14:13:03.016189 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.016064    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.016457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.016442    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sjnp6\""
Jul 19 14:13:03.016524 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.016488    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.018228 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.018214    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.019348 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.019435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019355    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-device-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.019435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-socket-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.019435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-registration-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.019565 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019445    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-etc-selinux\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.019565 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019496    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-sys-fs\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.019565 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019542    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sv2x4\""
Jul 19 14:13:03.019668 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.019554    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72x8k\" (UniqueName: \"kubernetes.io/projected/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-kube-api-access-72x8k\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.020186 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020170    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.020424 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020408    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 19 14:13:03.020485 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020431    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 19 14:13:03.020485 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020444    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.020598 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020565    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.020646 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020600    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.020730 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020714    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 19 14:13:03.020840 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020825    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.020962 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.020948    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gmqnx\""
Jul 19 14:13:03.022810 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.022791    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.022810 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.022800    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hp2lq\""
Jul 19 14:13:03.022950 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.022852    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:03.022950 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.022806    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 19 14:13:03.023981 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.023966    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:13:03.024052 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.024042    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"
Jul 19 14:13:03.025316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.025297    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:03.025390 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.025370    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:03.027168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.027151    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:13:03.027798 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.027783    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.030045 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.030028    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 19 14:13:03.030098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.030065    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.030193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.030177    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.030235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.030188    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sj7q7\""
Jul 19 14:13:03.030235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.030215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.032820 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.032799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.032905 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.032842    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 19 14:13:03.032905 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.032861    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:03.032983 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.032917    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tbpjs\""
Jul 19 14:13:03.032983 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.032923    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 19 14:13:03.032983 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.032969    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.033288 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.033271    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 19 14:13:03.033330 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.033296    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.033364 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.033329    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 19 14:13:03.033871 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.033857    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:13:03.035391 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.035375    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.038661 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.037632    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 19 14:13:03.038661 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.037714    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 19 14:13:03.038661 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.037736    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jbb7g\""
Jul 19 14:13:03.039610 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.039590    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.041435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.041417    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal"]
Jul 19 14:13:03.041523 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.041445    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal"]
Jul 19 14:13:03.041592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.041552    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.041907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.041878    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.042000 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.041908    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.042000 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.041923    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sd8wj\""
Jul 19 14:13:03.043719 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.043693    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 19 14:13:03.043863 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.043790    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 19 14:13:03.043863 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.043804    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cgnh6\""
Jul 19 14:13:03.044486 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.043915    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 19 14:13:03.045781 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.045764    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wllmw"
Jul 19 14:13:03.051714 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.051696    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wllmw"
Jul 19 14:13:03.113521 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.113508    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 19 14:13:03.119959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.119943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-modprobe-d\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120011 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.119964    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-host\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120011 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.119980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3110fbaa-0806-4fb1-b3f6-717530900da6-iptables-alerter-script\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.120011 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.119995    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-cni-netd\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120011 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120008    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-k8s-cni-cncf-io\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120023    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120049    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-cnibin\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120074    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-os-release\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-kubernetes\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-run\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-system-cni-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysctl-d\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92hvq\" (UniqueName: \"kubernetes.io/projected/00d596f8-f72b-4dfe-b309-753f5ed13380-kube-api-access-92hvq\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-ovnkube-config\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-etc-kubernetes\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-slash\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-run-ovn-kubernetes\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7hrm\" (UniqueName: \"kubernetes.io/projected/c6f99107-5ce9-4e02-925d-00c443973ba2-kube-api-access-f7hrm\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-device-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-kubelet\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120361    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-conf-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120374    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120374    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-device-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gspvk\" (UniqueName: \"kubernetes.io/projected/7265185a-f739-4655-ad0c-0590debb071b-kube-api-access-gspvk\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.120438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-cni-bin\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120438    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mhkb\" (UniqueName: \"kubernetes.io/projected/400828d7-6517-45cc-b7a8-e870444bb0a0-kube-api-access-8mhkb\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-kubelet\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-ovnkube-script-lib\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/42db642d-3fb7-4e9c-a3f2-e726a97bf514-serviceca\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-run-netns\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-systemd\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-system-cni-dir\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh66b\" (UniqueName: \"kubernetes.io/projected/8633cae0-26e7-4e0b-a312-084901853626-kube-api-access-kh66b\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120650    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-sys\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-var-lib-kubelet\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-systemd-units\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-etc-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.120874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-dbus\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120757    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-registration-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-sys-fs\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72x8k\" (UniqueName: \"kubernetes.io/projected/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-kube-api-access-72x8k\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120813    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-registration-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-cni-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-hostroot\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cda87231-5ef5-41b0-b0be-6ae5faff9144-agent-certs\") pod \"konnectivity-agent-2mfrj\" (UID: \"cda87231-5ef5-41b0-b0be-6ae5faff9144\") " pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-sys-fs\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysctl-conf\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45l7l\" (UniqueName: \"kubernetes.io/projected/3110fbaa-0806-4fb1-b3f6-717530900da6-kube-api-access-45l7l\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.120986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-multus-certs\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121012    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-hosts-file\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121028    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d596f8-f72b-4dfe-b309-753f5ed13380-tmp\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.121278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121053    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-node-log\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121069    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-cni-bin\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121083    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-env-overrides\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121148    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6f99107-5ce9-4e02-925d-00c443973ba2-ovn-node-metrics-cert\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-kubelet-config\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121183    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysconfig\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-lib-modules\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-tuned\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-cnibin\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-os-release\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cda87231-5ef5-41b0-b0be-6ae5faff9144-konnectivity-ca\") pod \"konnectivity-agent-2mfrj\" (UID: \"cda87231-5ef5-41b0-b0be-6ae5faff9144\") " pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qmr5\" (UniqueName: \"kubernetes.io/projected/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-kube-api-access-9qmr5\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121332    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-var-lib-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.121689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-ovn\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121385    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/400828d7-6517-45cc-b7a8-e870444bb0a0-cni-binary-copy\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-tmp-dir\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-systemd\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42db642d-3fb7-4e9c-a3f2-e726a97bf514-host\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121501    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-daemon-config\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3110fbaa-0806-4fb1-b3f6-717530900da6-host-slash\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-socket-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-etc-selinux\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121655    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-cni-binary-copy\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121690    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-socket-dir\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.122157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121729    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-etc-selinux\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.122526 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.122526 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-socket-dir-parent\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.122526 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-netns\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.122526 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-cni-multus\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.122526 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121914    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-log-socket\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.122526 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.121929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgtnv\" (UniqueName: \"kubernetes.io/projected/42db642d-3fb7-4e9c-a3f2-e726a97bf514-kube-api-access-jgtnv\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.129441 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.129424    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 19 14:13:03.132641 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.132625    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72x8k\" (UniqueName: \"kubernetes.io/projected/a13fa7b5-a9eb-410d-b69a-7cf0a40f75db-kube-api-access-72x8k\") pod \"aws-ebs-csi-driver-node-spnzc\" (UID: \"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.164235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.164190    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-18 14:08:02 +0000 UTC" deadline="2028-01-16 09:03:28.847780821 +0000 UTC"
Jul 19 14:13:03.164235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.164230    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13098h50m25.683552749s"
Jul 19 14:13:03.222850 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d596f8-f72b-4dfe-b309-753f5ed13380-tmp\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.222850 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-node-log\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222864    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-cni-bin\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-env-overrides\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6f99107-5ce9-4e02-925d-00c443973ba2-ovn-node-metrics-cert\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-kubelet-config\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysconfig\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222928    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-node-log\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.222976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222936    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-cni-bin\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.222981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysconfig\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223016    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-kubelet-config\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-lib-modules\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-tuned\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223150    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-cnibin\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-os-release\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223215    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-lib-modules\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cda87231-5ef5-41b0-b0be-6ae5faff9144-konnectivity-ca\") pod \"konnectivity-agent-2mfrj\" (UID: \"cda87231-5ef5-41b0-b0be-6ae5faff9144\") " pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qmr5\" (UniqueName: \"kubernetes.io/projected/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-kube-api-access-9qmr5\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.223295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-var-lib-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-ovn\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/400828d7-6517-45cc-b7a8-e870444bb0a0-cni-binary-copy\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223338    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-cnibin\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-tmp-dir\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-systemd\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223423    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-env-overrides\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42db642d-3fb7-4e9c-a3f2-e726a97bf514-host\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.223492    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-daemon-config\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3110fbaa-0806-4fb1-b3f6-717530900da6-host-slash\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.223829 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.223623    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:13:03.723580558 +0000 UTC m=+2.026411570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-cni-binary-copy\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-socket-dir-parent\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223723    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cda87231-5ef5-41b0-b0be-6ae5faff9144-konnectivity-ca\") pod \"konnectivity-agent-2mfrj\" (UID: \"cda87231-5ef5-41b0-b0be-6ae5faff9144\") " pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223758    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-var-lib-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-ovn\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223621    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3110fbaa-0806-4fb1-b3f6-717530900da6-host-slash\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.223869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-tmp-dir\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-netns\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-cni-multus\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-netns\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224113    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-log-socket\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgtnv\" (UniqueName: \"kubernetes.io/projected/42db642d-3fb7-4e9c-a3f2-e726a97bf514-kube-api-access-jgtnv\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.224479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224320    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-cni-multus\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-modprobe-d\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224375    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-systemd\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224376    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-host\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224421    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42db642d-3fb7-4e9c-a3f2-e726a97bf514-host\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3110fbaa-0806-4fb1-b3f6-717530900da6-iptables-alerter-script\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224463    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-cni-netd\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224612    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-cni-binary-copy\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224677    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-k8s-cni-cncf-io\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224680    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-host\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224730    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/400828d7-6517-45cc-b7a8-e870444bb0a0-cni-binary-copy\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224762    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-modprobe-d\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-os-release\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.225233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-socket-dir-parent\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224838    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-cni-netd\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-log-socket\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224961    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3110fbaa-0806-4fb1-b3f6-717530900da6-iptables-alerter-script\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.224982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-k8s-cni-cncf-io\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.225054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.225552    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-daemon-config\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.225608    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d596f8-f72b-4dfe-b309-753f5ed13380-tmp\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.225656    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6f99107-5ce9-4e02-925d-00c443973ba2-ovn-node-metrics-cert\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.225939    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.225992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.225989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-cnibin\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226014    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-os-release\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-kubernetes\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-run\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-system-cni-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226091    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-cnibin\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysctl-d\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226118    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-os-release\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92hvq\" (UniqueName: \"kubernetes.io/projected/00d596f8-f72b-4dfe-b309-753f5ed13380-kube-api-access-92hvq\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-system-cni-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-ovnkube-config\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226185    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-etc-kubernetes\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-run\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226118    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-kubernetes\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226187    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysctl-d\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.226457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-slash\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-run-ovn-kubernetes\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226258    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-slash\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226265    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-etc-kubernetes\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7hrm\" (UniqueName: \"kubernetes.io/projected/c6f99107-5ce9-4e02-925d-00c443973ba2-kube-api-access-f7hrm\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226304    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-tuned\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-kubelet\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226359    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-run-ovn-kubernetes\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-conf-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gspvk\" (UniqueName: \"kubernetes.io/projected/7265185a-f739-4655-ad0c-0590debb071b-kube-api-access-gspvk\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226402    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-kubelet\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-cni-bin\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mhkb\" (UniqueName: \"kubernetes.io/projected/400828d7-6517-45cc-b7a8-e870444bb0a0-kube-api-access-8mhkb\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227157 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-conf-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-var-lib-cni-bin\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-kubelet\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226523    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-ovnkube-script-lib\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226525    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-kubelet\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226570    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/42db642d-3fb7-4e9c-a3f2-e726a97bf514-serviceca\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-run-netns\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.226613    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226617    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-systemd\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-system-cni-dir\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.226667    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:13:03.726651584 +0000 UTC m=+2.029482580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226685    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8633cae0-26e7-4e0b-a312-084901853626-system-cni-dir\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh66b\" (UniqueName: \"kubernetes.io/projected/8633cae0-26e7-4e0b-a312-084901853626-kube-api-access-kh66b\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.227700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-sys\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-var-lib-kubelet\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226742    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-run-systemd\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-var-lib-kubelet\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-systemd-units\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226796    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-ovnkube-config\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226808    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8633cae0-26e7-4e0b-a312-084901853626-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226823    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-etc-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-host-run-netns\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226847    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-dbus\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226859    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-etc-openvswitch\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226872    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-cni-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226881    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-sys\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226823    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6f99107-5ce9-4e02-925d-00c443973ba2-systemd-units\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-hostroot\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.228078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cda87231-5ef5-41b0-b0be-6ae5faff9144-agent-certs\") pod \"konnectivity-agent-2mfrj\" (UID: \"cda87231-5ef5-41b0-b0be-6ae5faff9144\") " pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysctl-conf\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226979    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-dbus\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45l7l\" (UniqueName: \"kubernetes.io/projected/3110fbaa-0806-4fb1-b3f6-717530900da6-kube-api-access-45l7l\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226987    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-hostroot\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.226956    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-multus-cni-dir\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-multus-certs\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-hosts-file\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227053    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6f99107-5ce9-4e02-925d-00c443973ba2-ovnkube-script-lib\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227086    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/42db642d-3fb7-4e9c-a3f2-e726a97bf514-serviceca\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227102    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/400828d7-6517-45cc-b7a8-e870444bb0a0-host-run-multus-certs\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227106    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-hosts-file\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.228507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.227104    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d596f8-f72b-4dfe-b309-753f5ed13380-etc-sysctl-conf\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.229219 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.229202    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cda87231-5ef5-41b0-b0be-6ae5faff9144-agent-certs\") pod \"konnectivity-agent-2mfrj\" (UID: \"cda87231-5ef5-41b0-b0be-6ae5faff9144\") " pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.231770 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.231752    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:03.231836 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.231774    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:03.231836 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.231788    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:03.231954 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.231851    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:03.731835484 +0000 UTC m=+2.034666484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:03.233114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.233094    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qmr5\" (UniqueName: \"kubernetes.io/projected/76b4d6b2-022a-48a5-a1ca-195f08d99cf3-kube-api-access-9qmr5\") pod \"node-resolver-597hr\" (UID: \"76b4d6b2-022a-48a5-a1ca-195f08d99cf3\") " pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.234120 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.234098    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgtnv\" (UniqueName: \"kubernetes.io/projected/42db642d-3fb7-4e9c-a3f2-e726a97bf514-kube-api-access-jgtnv\") pod \"node-ca-kxf6j\" (UID: \"42db642d-3fb7-4e9c-a3f2-e726a97bf514\") " pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.234841 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.234817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gspvk\" (UniqueName: \"kubernetes.io/projected/7265185a-f739-4655-ad0c-0590debb071b-kube-api-access-gspvk\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.235072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.235051    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92hvq\" (UniqueName: \"kubernetes.io/projected/00d596f8-f72b-4dfe-b309-753f5ed13380-kube-api-access-92hvq\") pod \"tuned-4j4bz\" (UID: \"00d596f8-f72b-4dfe-b309-753f5ed13380\") " pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.235269 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.235252    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mhkb\" (UniqueName: \"kubernetes.io/projected/400828d7-6517-45cc-b7a8-e870444bb0a0-kube-api-access-8mhkb\") pod \"multus-mkk5t\" (UID: \"400828d7-6517-45cc-b7a8-e870444bb0a0\") " pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.236026 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.236011    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7hrm\" (UniqueName: \"kubernetes.io/projected/c6f99107-5ce9-4e02-925d-00c443973ba2-kube-api-access-f7hrm\") pod \"ovnkube-node-h4r72\" (UID: \"c6f99107-5ce9-4e02-925d-00c443973ba2\") " pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.238284 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.238263    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45l7l\" (UniqueName: \"kubernetes.io/projected/3110fbaa-0806-4fb1-b3f6-717530900da6-kube-api-access-45l7l\") pod \"iptables-alerter-rsxw5\" (UID: \"3110fbaa-0806-4fb1-b3f6-717530900da6\") " pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.241223 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.241207    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh66b\" (UniqueName: \"kubernetes.io/projected/8633cae0-26e7-4e0b-a312-084901853626-kube-api-access-kh66b\") pod \"multus-additional-cni-plugins-ssfg4\" (UID: \"8633cae0-26e7-4e0b-a312-084901853626\") " pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.343248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.343206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc"
Jul 19 14:13:03.357355 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.357336    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4j4bz"
Jul 19 14:13:03.364135 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.364117    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ssfg4"
Jul 19 14:13:03.377627 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.377611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mkk5t"
Jul 19 14:13:03.391289 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.391270    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rsxw5"
Jul 19 14:13:03.396083 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.396060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:03.398644 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.398628    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:03.427701 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.427678    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-597hr"
Jul 19 14:13:03.431769 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.431754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxf6j"
Jul 19 14:13:03.497076 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.497040    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7dc26ae2909348f6daa2c1d449c4ac5.slice/crio-7701503ba462899732589dd290a496b9d31752327e4edba3261342d15046ecbf WatchSource:0}: Error finding container 7701503ba462899732589dd290a496b9d31752327e4edba3261342d15046ecbf: Status 404 returned error can't find the container with id 7701503ba462899732589dd290a496b9d31752327e4edba3261342d15046ecbf
Jul 19 14:13:03.497461 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.497441    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabed4ba2b3d6a2cc98e02bb7bec66ad.slice/crio-8029fed0059e7675813d1dfdfe6f5f6f0bf810874af7b6c5c769ce66a56abbdb WatchSource:0}: Error finding container 8029fed0059e7675813d1dfdfe6f5f6f0bf810874af7b6c5c769ce66a56abbdb: Status 404 returned error can't find the container with id 8029fed0059e7675813d1dfdfe6f5f6f0bf810874af7b6c5c769ce66a56abbdb
Jul 19 14:13:03.500830 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.500816    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:13:03.569256 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.569220    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:03.666877 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.666855    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13fa7b5_a9eb_410d_b69a_7cf0a40f75db.slice/crio-1db43a3947a166b110b4dd5fbc89fd24d45e82c82b6b1c836dccc6c0f2974579 WatchSource:0}: Error finding container 1db43a3947a166b110b4dd5fbc89fd24d45e82c82b6b1c836dccc6c0f2974579: Status 404 returned error can't find the container with id 1db43a3947a166b110b4dd5fbc89fd24d45e82c82b6b1c836dccc6c0f2974579
Jul 19 14:13:03.695382 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.695361    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8633cae0_26e7_4e0b_a312_084901853626.slice/crio-957aebd375ef2ba76a1f3752c597d50c00131268d6bb6dab9469780eb0551b5d WatchSource:0}: Error finding container 957aebd375ef2ba76a1f3752c597d50c00131268d6bb6dab9469780eb0551b5d: Status 404 returned error can't find the container with id 957aebd375ef2ba76a1f3752c597d50c00131268d6bb6dab9469780eb0551b5d
Jul 19 14:13:03.728400 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.728379    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d596f8_f72b_4dfe_b309_753f5ed13380.slice/crio-a007a35056cff80df6a9b478d80eca052a0c7093d175f7cbb2e73a3374333363 WatchSource:0}: Error finding container a007a35056cff80df6a9b478d80eca052a0c7093d175f7cbb2e73a3374333363: Status 404 returned error can't find the container with id a007a35056cff80df6a9b478d80eca052a0c7093d175f7cbb2e73a3374333363
Jul 19 14:13:03.730233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.730219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:03.730289 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.730262    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:03.730379 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.730364    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:03.730439 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.730428    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:13:04.730408803 +0000 UTC m=+3.033239801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:03.730502 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.730367    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:03.730502 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.730492    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:13:04.730480307 +0000 UTC m=+3.033311306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:03.773625 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.773603    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42db642d_3fb7_4e9c_a3f2_e726a97bf514.slice/crio-f1d2fb2355b7fff367a71cd1aedee0d4e165b6e83be2becbf0edd09f2991c15c WatchSource:0}: Error finding container f1d2fb2355b7fff367a71cd1aedee0d4e165b6e83be2becbf0edd09f2991c15c: Status 404 returned error can't find the container with id f1d2fb2355b7fff367a71cd1aedee0d4e165b6e83be2becbf0edd09f2991c15c
Jul 19 14:13:03.776463 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.776383    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3110fbaa_0806_4fb1_b3f6_717530900da6.slice/crio-c2adb6fe30144d5bc6729f72b380d3894bae8cfc18c5052d5e1868ceb13e4a56 WatchSource:0}: Error finding container c2adb6fe30144d5bc6729f72b380d3894bae8cfc18c5052d5e1868ceb13e4a56: Status 404 returned error can't find the container with id c2adb6fe30144d5bc6729f72b380d3894bae8cfc18c5052d5e1868ceb13e4a56
Jul 19 14:13:03.776915 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.776892    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda87231_5ef5_41b0_b0be_6ae5faff9144.slice/crio-ae8216dc415e8840db04e55554674c6394fc345cb1ffd51b18a61c7ad4504aa9 WatchSource:0}: Error finding container ae8216dc415e8840db04e55554674c6394fc345cb1ffd51b18a61c7ad4504aa9: Status 404 returned error can't find the container with id ae8216dc415e8840db04e55554674c6394fc345cb1ffd51b18a61c7ad4504aa9
Jul 19 14:13:03.795738 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.795719    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b4d6b2_022a_48a5_a1ca_195f08d99cf3.slice/crio-87d1bb2579e561273040f197b1cb2d1ecf18f77b57c11b36f8d246e1a55614eb WatchSource:0}: Error finding container 87d1bb2579e561273040f197b1cb2d1ecf18f77b57c11b36f8d246e1a55614eb: Status 404 returned error can't find the container with id 87d1bb2579e561273040f197b1cb2d1ecf18f77b57c11b36f8d246e1a55614eb
Jul 19 14:13:03.831000 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:03.830981    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:03.831105 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.831093    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:03.831157 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.831109    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:03.831157 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.831117    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:03.831251 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:03.831159    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:04.831143175 +0000 UTC m=+3.133974182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:03.831884 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.831862    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f99107_5ce9_4e02_925d_00c443973ba2.slice/crio-8a72d3cca1af9adfbd7a362474a7c337d2d0132066c44b8462758c20b623f25b WatchSource:0}: Error finding container 8a72d3cca1af9adfbd7a362474a7c337d2d0132066c44b8462758c20b623f25b: Status 404 returned error can't find the container with id 8a72d3cca1af9adfbd7a362474a7c337d2d0132066c44b8462758c20b623f25b
Jul 19 14:13:03.940126 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:03.940093    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400828d7_6517_45cc_b7a8_e870444bb0a0.slice/crio-23251f41ba7ac5b313ceb18b41ca643043b202a8bfd6fb3d2d1f8ec9b26f4d62 WatchSource:0}: Error finding container 23251f41ba7ac5b313ceb18b41ca643043b202a8bfd6fb3d2d1f8ec9b26f4d62: Status 404 returned error can't find the container with id 23251f41ba7ac5b313ceb18b41ca643043b202a8bfd6fb3d2d1f8ec9b26f4d62
Jul 19 14:13:04.055280 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.055224    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:08:03 +0000 UTC" deadline="2028-03-30 17:57:46.61954793 +0000 UTC"
Jul 19 14:13:04.055280 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.055280    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14883h44m42.564272197s"
Jul 19 14:13:04.247061 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.246994    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:04.392459 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.392371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:04.392642 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.392486    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:04.392893 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.392873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:04.393015 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.392992    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:04.403586 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.403538    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"8a72d3cca1af9adfbd7a362474a7c337d2d0132066c44b8462758c20b623f25b"}
Jul 19 14:13:04.410549 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.410506    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rsxw5" event={"ID":"3110fbaa-0806-4fb1-b3f6-717530900da6","Type":"ContainerStarted","Data":"c2adb6fe30144d5bc6729f72b380d3894bae8cfc18c5052d5e1868ceb13e4a56"}
Jul 19 14:13:04.418976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.418897    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4j4bz" event={"ID":"00d596f8-f72b-4dfe-b309-753f5ed13380","Type":"ContainerStarted","Data":"a007a35056cff80df6a9b478d80eca052a0c7093d175f7cbb2e73a3374333363"}
Jul 19 14:13:04.427037 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.427011    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc" event={"ID":"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db","Type":"ContainerStarted","Data":"1db43a3947a166b110b4dd5fbc89fd24d45e82c82b6b1c836dccc6c0f2974579"}
Jul 19 14:13:04.445600 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.445429    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mkk5t" event={"ID":"400828d7-6517-45cc-b7a8-e870444bb0a0","Type":"ContainerStarted","Data":"23251f41ba7ac5b313ceb18b41ca643043b202a8bfd6fb3d2d1f8ec9b26f4d62"}
Jul 19 14:13:04.464690 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.464360    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-597hr" event={"ID":"76b4d6b2-022a-48a5-a1ca-195f08d99cf3","Type":"ContainerStarted","Data":"87d1bb2579e561273040f197b1cb2d1ecf18f77b57c11b36f8d246e1a55614eb"}
Jul 19 14:13:04.485022 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.484945    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2mfrj" event={"ID":"cda87231-5ef5-41b0-b0be-6ae5faff9144","Type":"ContainerStarted","Data":"ae8216dc415e8840db04e55554674c6394fc345cb1ffd51b18a61c7ad4504aa9"}
Jul 19 14:13:04.500756 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.500676    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxf6j" event={"ID":"42db642d-3fb7-4e9c-a3f2-e726a97bf514","Type":"ContainerStarted","Data":"f1d2fb2355b7fff367a71cd1aedee0d4e165b6e83be2becbf0edd09f2991c15c"}
Jul 19 14:13:04.502389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.502323    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerStarted","Data":"957aebd375ef2ba76a1f3752c597d50c00131268d6bb6dab9469780eb0551b5d"}
Jul 19 14:13:04.513842 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.513783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal" event={"ID":"a7dc26ae2909348f6daa2c1d449c4ac5","Type":"ContainerStarted","Data":"7701503ba462899732589dd290a496b9d31752327e4edba3261342d15046ecbf"}
Jul 19 14:13:04.529875 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.529809    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal" event={"ID":"eabed4ba2b3d6a2cc98e02bb7bec66ad","Type":"ContainerStarted","Data":"8029fed0059e7675813d1dfdfe6f5f6f0bf810874af7b6c5c769ce66a56abbdb"}
Jul 19 14:13:04.714758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.714728    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:04.741570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.739455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:04.741570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.739557    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:04.741570 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.739704    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:04.741570 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.739767    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:13:06.73974987 +0000 UTC m=+5.042580868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:04.741570 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.740144    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:04.741570 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.740194    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:13:06.740179107 +0000 UTC m=+5.043010107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:04.841661 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:04.841562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:04.841824 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.841745    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:04.841824 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.841771    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:04.841824 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.841785    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:04.841980 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:04.841843    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:06.841824955 +0000 UTC m=+5.144655960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:05.056131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:05.056061    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:08:03 +0000 UTC" deadline="2028-02-11 06:39:13.455648037 +0000 UTC"
Jul 19 14:13:05.056131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:05.056099    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13720h26m8.399554487s"
Jul 19 14:13:05.204414 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:05.204348    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:13:05.391726 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:05.391694    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:05.391908 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:05.391843    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:06.393087 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:06.393010    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:06.393976 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.393151    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:06.393976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:06.393216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:06.393976 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.393284    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:06.759871 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:06.759534    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:06.759871 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:06.759609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:06.759871 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.759667    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:06.759871 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.759750    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:06.759871 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.759767    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:13:10.759746748 +0000 UTC m=+9.062577763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:06.759871 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.759802    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:13:10.759786953 +0000 UTC m=+9.062617955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:06.860702 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:06.860664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:06.860922 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.860882    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:06.860922 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.860905    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:06.860922 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.860917    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:06.861123 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:06.860977    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:10.860958237 +0000 UTC m=+9.163789234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:07.391535 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:07.391497    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:07.391750 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:07.391670    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:08.391738 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:08.391694    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:08.392247 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:08.391851    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:08.392247 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:08.392244    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:08.392368 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:08.392334    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:09.391108 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:09.391076    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:09.391275 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:09.391212    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:10.391585 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:10.391028    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:10.391585 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:10.391156    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:10.391585 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.391183    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:10.391585 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.391235    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:10.793490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:10.793411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:10.793490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:10.793477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:10.793723 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.793609    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:10.793723 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.793669    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:13:18.793652547 +0000 UTC m=+17.096483561 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:10.793723 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.793608    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:10.793873 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.793787    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:13:18.793766993 +0000 UTC m=+17.096598001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:10.894669 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:10.894627    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:10.894838 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.894811    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:10.894838 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.894836    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:10.894948 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.894848    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:10.894948 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:10.894914    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:18.894895401 +0000 UTC m=+17.197726400 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:11.391312 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:11.391277    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:11.391489 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:11.391427    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:12.393146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:12.392488    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:12.393146 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:12.392623    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:12.393146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:12.393018    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:12.393146 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:12.393104    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:13.391470 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:13.391437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:13.391680 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:13.391571    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:14.391505 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:14.391467    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:14.391991 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:14.391624    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:14.391991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:14.391675    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:14.391991 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:14.391804    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:15.391120 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:15.391086    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:15.391296 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:15.391194    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:16.390894 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:16.390853    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:16.391318 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:16.390996    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:16.391318 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:16.391046    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:16.391318 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:16.391161    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:17.390745 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:17.390708    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:17.390904 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:17.390857    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:18.391464 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:18.391425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:18.391932 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:18.391425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:18.391932 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.391580    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:18.391932 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.391652    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:18.858189 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:18.858112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:18.858336 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:18.858189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:18.858336 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.858275    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:18.858427 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.858347    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:13:34.85832848 +0000 UTC m=+33.161159491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:18.858427 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.858280    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:18.858427 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.858413    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:13:34.858397311 +0000 UTC m=+33.161228325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:18.958816 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:18.958780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:18.958946 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.958908    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:18.958946 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.958922    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:18.958946 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.958930    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:18.959051 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:18.958972    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:34.958960951 +0000 UTC m=+33.261791945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:19.390870 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:19.390829    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:19.391057 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:19.390968    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:20.391061 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:20.391025    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:20.391506 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:20.391177    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:20.391506 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:20.391234    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:20.391506 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:20.391356    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:21.390880 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:21.390703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:21.391052 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:21.390967    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:22.391654 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:22.391619    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:22.392041 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:22.391793    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:22.392041 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:22.391812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:22.392041 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:22.391908    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:23.391012 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.390986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:23.391148 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:23.391125    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:23.579683 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.579469    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal" event={"ID":"eabed4ba2b3d6a2cc98e02bb7bec66ad","Type":"ContainerStarted","Data":"0cba881e5291058c451cfd64724210eba9f375c45c9460501d689618cde12fbb"}
Jul 19 14:13:23.581786 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.581769    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:13:23.582139 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.582113    2574 generic.go:356] "Generic (PLEG): container finished" podID="c6f99107-5ce9-4e02-925d-00c443973ba2" containerID="37235f684896dbc47610285ca1db4434d5f84268c7494320244f56cc27f74ee2" exitCode=1
Jul 19 14:13:23.582197 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.582177    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"9dd802a9ece419fda7b26d3b061d134d93d9d7ae88065505dcf49c4218fa3fb7"}
Jul 19 14:13:23.582256 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.582245    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerDied","Data":"37235f684896dbc47610285ca1db4434d5f84268c7494320244f56cc27f74ee2"}
Jul 19 14:13:23.582303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.582282    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"a86a803a3c7a19d206949e869750ce4976601cc76eb75a931455a91bb31cf237"}
Jul 19 14:13:23.583823 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.583799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4j4bz" event={"ID":"00d596f8-f72b-4dfe-b309-753f5ed13380","Type":"ContainerStarted","Data":"8fd4d6ccbe70df47a99aea7d21c1bdcb1a3eb2a16e63a118122b71f357041aef"}
Jul 19 14:13:23.585252 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.585229    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mkk5t" event={"ID":"400828d7-6517-45cc-b7a8-e870444bb0a0","Type":"ContainerStarted","Data":"32ead962e435912f053ee5842862132734a378ee5276344ec2b202055e54f983"}
Jul 19 14:13:23.597049 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.597008    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-90.ec2.internal" podStartSLOduration=20.59699564 podStartE2EDuration="20.59699564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:23.596562009 +0000 UTC m=+21.899393024" watchObservedRunningTime="2026-07-19 14:13:23.59699564 +0000 UTC m=+21.899826655"
Jul 19 14:13:23.627435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.627391    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4j4bz" podStartSLOduration=2.301931635 podStartE2EDuration="21.62737741s" totalImagesPullingTime="19.325445775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:23.610729641 +0000 UTC m=+21.913560656" watchObservedRunningTime="2026-07-19 14:13:23.62737741 +0000 UTC m=+21.930208424"
Jul 19 14:13:23.627632 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:23.627611    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mkk5t" podStartSLOduration=2.159543413 podStartE2EDuration="21.627603723s" totalImagesPullingTime="19.46806031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:23.627334324 +0000 UTC m=+21.930165354" watchObservedRunningTime="2026-07-19 14:13:23.627603723 +0000 UTC m=+21.930434739"
Jul 19 14:13:24.391630 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.391596    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:24.391804 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.391597    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:24.391804 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:24.391717    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:24.391804 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:24.391793    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:24.589743 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.588805    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-597hr" event={"ID":"76b4d6b2-022a-48a5-a1ca-195f08d99cf3","Type":"ContainerStarted","Data":"5b960b8ff09f1f1aac69fdeba9c1953a9eea130f177f131f58e9b86d2ba62319"}
Jul 19 14:13:24.591959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.591917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2mfrj" event={"ID":"cda87231-5ef5-41b0-b0be-6ae5faff9144","Type":"ContainerStarted","Data":"59cddc9f5758aafc505f835a61cc7b49352baeccfe46a58aa6862cda8e39486e"}
Jul 19 14:13:24.594121 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.594098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxf6j" event={"ID":"42db642d-3fb7-4e9c-a3f2-e726a97bf514","Type":"ContainerStarted","Data":"30c1a4982909701f62c19b9bd08507be377a0297893965a889e1ecf28e304892"}
Jul 19 14:13:24.595686 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.595661    2574 generic.go:356] "Generic (PLEG): container finished" podID="8633cae0-26e7-4e0b-a312-084901853626" containerID="16d3b3b7936cd8eb4c9c8f5c24e5f78925b1867958c30d3a3ceb36f1a91e5405" exitCode=0
Jul 19 14:13:24.595820 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.595743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerDied","Data":"16d3b3b7936cd8eb4c9c8f5c24e5f78925b1867958c30d3a3ceb36f1a91e5405"}
Jul 19 14:13:24.597482 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.597460    2574 generic.go:356] "Generic (PLEG): container finished" podID="a7dc26ae2909348f6daa2c1d449c4ac5" containerID="38995fe55a5e0f4a86b942d3666632b080ad752974d75fd78691e46b33b0bfcd" exitCode=0
Jul 19 14:13:24.597640 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.597557    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal" event={"ID":"a7dc26ae2909348f6daa2c1d449c4ac5","Type":"ContainerDied","Data":"38995fe55a5e0f4a86b942d3666632b080ad752974d75fd78691e46b33b0bfcd"}
Jul 19 14:13:24.600748 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.600729    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:13:24.601161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.601142    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"4880f6abd83853e2baf52cf4ed998e498a880f08a8da4f2b2cbf9a4a3a1dc904"}
Jul 19 14:13:24.601339 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.601300    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"69a359eddb43a8bd1dfee597fda26fc0a697465d7a862af954512ad75290be7c"}
Jul 19 14:13:24.601424 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.601407    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"30ac4ef18d45361c26a90e257d3c98ce903e5f909ec74f46d7ecb85a26ba66da"}
Jul 19 14:13:24.603208 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.603182    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rsxw5" event={"ID":"3110fbaa-0806-4fb1-b3f6-717530900da6","Type":"ContainerStarted","Data":"68913a357aa0f21feb0be9fb6c90087da13ebb68bb60458a53c798186dcbcdeb"}
Jul 19 14:13:24.604119 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.604080    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-597hr" podStartSLOduration=3.340382046 podStartE2EDuration="22.604067041s" totalImagesPullingTime="19.263684995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:24.603419201 +0000 UTC m=+22.906250217" watchObservedRunningTime="2026-07-19 14:13:24.604067041 +0000 UTC m=+22.906898059"
Jul 19 14:13:24.604760 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.604738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc" event={"ID":"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db","Type":"ContainerStarted","Data":"bc2610fd88b58f839dade74627c675a78680f6d5babe124c6fffe54f9ee93609"}
Jul 19 14:13:24.649400 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.649163    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kxf6j" podStartSLOduration=11.529776935 podStartE2EDuration="22.649144622s" totalImagesPullingTime="11.119367687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:24.648887533 +0000 UTC m=+22.951718549" watchObservedRunningTime="2026-07-19 14:13:24.649144622 +0000 UTC m=+22.951975639"
Jul 19 14:13:24.662481 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.662432    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rsxw5" podStartSLOduration=3.386665527 podStartE2EDuration="22.662412855s" totalImagesPullingTime="19.275747328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:24.662325711 +0000 UTC m=+22.965156728" watchObservedRunningTime="2026-07-19 14:13:24.662412855 +0000 UTC m=+22.965243872"
Jul 19 14:13:24.725420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.725373    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:24.806742 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.806710    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:24.859815 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:24.859721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:25.391108 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:25.391081    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:25.391243 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:25.391196    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:25.450008 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:25.449964    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 19 14:13:25.610169 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:25.610080    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc" event={"ID":"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db","Type":"ContainerStarted","Data":"63f8af80ecfd4ed8ad20d5657c00b3363d1cdad147d40d22d70d00cf2b3a4770"}
Jul 19 14:13:25.611954 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:25.611917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal" event={"ID":"a7dc26ae2909348f6daa2c1d449c4ac5","Type":"ContainerStarted","Data":"e2573bed1a254d6e953e6fbd97180577f7b5464e5cbb870ae1b08262796285e4"}
Jul 19 14:13:25.625745 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:25.625697    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2mfrj" podStartSLOduration=4.349623933 podStartE2EDuration="23.625676941s" totalImagesPullingTime="19.276053008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:24.675943437 +0000 UTC m=+22.978774453" watchObservedRunningTime="2026-07-19 14:13:25.625676941 +0000 UTC m=+23.928507957"
Jul 19 14:13:25.625915 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:25.625892    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-90.ec2.internal" podStartSLOduration=22.625884895 podStartE2EDuration="22.625884895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:25.625387999 +0000 UTC m=+23.928219016" watchObservedRunningTime="2026-07-19 14:13:25.625884895 +0000 UTC m=+23.928715910"
Jul 19 14:13:26.216502 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.216464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:26.218139 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.218032    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-19T14:13:25.450005324Z","UUID":"965532d5-7ce8-4939-9aaa-66d58b20fddc","Handler":null,"Name":"","Endpoint":""}
Jul 19 14:13:26.219654 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.219631    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 19 14:13:26.219771 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.219673    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 19 14:13:26.317200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.317126    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:26.391549 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.391509    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:26.391789 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:26.391645    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:26.391789 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.391764    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:26.391897 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:26.391867    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:26.482611 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.482576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:26.582113 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.582022    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:26.618389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.618360    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:13:26.618866 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.618842    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"4b2670f2e207d908b1dbd6a5e6934ceec6cccd90896174c15793edf74059a696"}
Jul 19 14:13:26.826751 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:26.826721    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:27.199763 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.199731    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:27.300098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.300064    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:27.391377 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.391341    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:27.391570 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:27.391462    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:27.563101 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.563022    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:27.622481 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.622451    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc" event={"ID":"a13fa7b5-a9eb-410d-b69a-7cf0a40f75db","Type":"ContainerStarted","Data":"f2084be799217ad134f7798927a8b25fa9e154e8d2e540b03e755e5f0f359190"}
Jul 19 14:13:27.634060 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.634030    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:27.639149 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.639108    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-spnzc" podStartSLOduration=3.803057755 podStartE2EDuration="25.639094193s" totalImagesPullingTime="21.836036438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:27.639024363 +0000 UTC m=+25.941855381" watchObservedRunningTime="2026-07-19 14:13:27.639094193 +0000 UTC m=+25.941925207"
Jul 19 14:13:27.735532 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.735487    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:27.971291 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.971213    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:27.971986 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:27.971963    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:28.391692 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:28.391613    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:28.391834 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:28.391613    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:28.391834 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:28.391737    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:28.391834 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:28.391808    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:28.625338 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:28.625312    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2mfrj"
Jul 19 14:13:28.682682 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:28.680573    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:29.390863 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.390698    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:29.391013 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:29.390943    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:29.628376 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.628341    2574 generic.go:356] "Generic (PLEG): container finished" podID="8633cae0-26e7-4e0b-a312-084901853626" containerID="d9ec0fc215dbf66e6254a9430bfb2e3ff53c3c6e88bdf608a2636897078cfc44" exitCode=0
Jul 19 14:13:29.629012 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.628408    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerDied","Data":"d9ec0fc215dbf66e6254a9430bfb2e3ff53c3c6e88bdf608a2636897078cfc44"}
Jul 19 14:13:29.632009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.631985    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:13:29.632329 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.632311    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"2cbbd26eeb1c9e830453f16e1ff3f0306288e7b2f2f82e2ba6db36b77c4e69fe"}
Jul 19 14:13:29.632571 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.632496    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:29.632637 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.632574    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:29.632637 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.632623    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:29.632809 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.632747    2574 scope.go:117] "RemoveContainer" containerID="37235f684896dbc47610285ca1db4434d5f84268c7494320244f56cc27f74ee2"
Jul 19 14:13:29.647402 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.647384    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:29.647882 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:29.647866    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:13:30.143487 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.143454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:30.353291 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.353215    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:30.391999 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.391498    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:30.391999 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.391510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:30.391999 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:30.391648    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:30.391999 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:30.391780    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:30.454540 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.454210    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:30.641756 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.641688    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:13:30.642390 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.642079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" event={"ID":"c6f99107-5ce9-4e02-925d-00c443973ba2","Type":"ContainerStarted","Data":"0856bb6ed2ac88165b0c25d86d7234f68a1a7f3a9798aff4e1fb657180b64772"}
Jul 19 14:13:30.667722 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:30.667661    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72" podStartSLOduration=9.419300573 podStartE2EDuration="28.667641364s" totalImagesPullingTime="19.248340791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:30.667480065 +0000 UTC m=+28.970311103" watchObservedRunningTime="2026-07-19 14:13:30.667641364 +0000 UTC m=+28.970472381"
Jul 19 14:13:31.234952 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.234753    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x7d24"]
Jul 19 14:13:31.235111 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.235039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:31.235149 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:31.235121    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:31.238229 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.238203    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cdvtc"]
Jul 19 14:13:31.238354 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.238316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:31.238419 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:31.238404    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:31.238912 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.238886    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6kfzr"]
Jul 19 14:13:31.239029 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.239001    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:31.239126 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:31.239106    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:31.645921 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.645892    2574 generic.go:356] "Generic (PLEG): container finished" podID="8633cae0-26e7-4e0b-a312-084901853626" containerID="805cd234b833b4e5232167e1f735792a37fe941f45be19176c8d4c3fa047ad69" exitCode=0
Jul 19 14:13:31.646286 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:31.645977    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerDied","Data":"805cd234b833b4e5232167e1f735792a37fe941f45be19176c8d4c3fa047ad69"}
Jul 19 14:13:32.396112 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:32.396075    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:32.396300 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:32.396188    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:32.396300 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:32.396222    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:32.396418 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:32.396343    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:33.180764 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.180691    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:33.290649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.290608    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:33.390807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.390781    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:33.390964 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:33.390870    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:33.652658 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.652583    2574 generic.go:356] "Generic (PLEG): container finished" podID="8633cae0-26e7-4e0b-a312-084901853626" containerID="b6b7e0fb63119f5ad22d419684f2bed435c1d884a8f9c16a2054b52064968acf" exitCode=0
Jul 19 14:13:33.652658 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.652635    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerDied","Data":"b6b7e0fb63119f5ad22d419684f2bed435c1d884a8f9c16a2054b52064968acf"}
Jul 19 14:13:33.865089 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.865059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:33.920959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:33.920887    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.309151 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.309078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.391372 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.391343    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:34.391372 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.391345    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:34.391610 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.391570    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7d24" podUID="ced506fe-e66e-4a7f-9fed-f1c72377b3ec"
Jul 19 14:13:34.391610 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.391569    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kfzr" podUID="7265185a-f739-4655-ad0c-0590debb071b"
Jul 19 14:13:34.406266 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.406238    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.505309 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.505279    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.605199 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.605118    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.740217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.740175    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.838931 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.838888    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-597hr_76b4d6b2-022a-48a5-a1ca-195f08d99cf3/dns-node-resolver/0.log"
Jul 19 14:13:34.839201 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.839172    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:34.877182 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.877108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:34.877182 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.877163    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:34.877453 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.877280    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:34.877453 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.877285    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:34.877453 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.877349    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret podName:ced506fe-e66e-4a7f-9fed-f1c72377b3ec nodeName:}" failed. No retries permitted until 2026-07-19 14:14:06.877331845 +0000 UTC m=+65.180162856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret") pod "global-pull-secret-syncer-x7d24" (UID: "ced506fe-e66e-4a7f-9fed-f1c72377b3ec") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:13:34.877453 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.877369    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs podName:7265185a-f739-4655-ad0c-0590debb071b nodeName:}" failed. No retries permitted until 2026-07-19 14:14:06.877359666 +0000 UTC m=+65.180190667 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs") pod "network-metrics-daemon-6kfzr" (UID: "7265185a-f739-4655-ad0c-0590debb071b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:13:34.978282 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:34.978250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:34.978433 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.978398    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:13:34.978433 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.978414    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:13:34.978433 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.978424    2574 projected.go:196] Error preparing data for projected volume kube-api-access-gwg7j for pod openshift-network-diagnostics/network-check-target-cdvtc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:34.978566 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:34.978472    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j podName:f8ae77e0-a2cd-4845-964e-a9709112c5a8 nodeName:}" failed. No retries permitted until 2026-07-19 14:14:06.978459937 +0000 UTC m=+65.281290932 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gwg7j" (UniqueName: "kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j") pod "network-check-target-cdvtc" (UID: "f8ae77e0-a2cd-4845-964e-a9709112c5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:13:35.012681 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:35.012645    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:35.115767 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:35.115736    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:35.390824 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:35.390796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:35.391242 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:35.390904    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdvtc" podUID="f8ae77e0-a2cd-4845-964e-a9709112c5a8"
Jul 19 14:13:35.496824 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:35.496795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:35.555404 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:35.555372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:36.020974 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.020940    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-90.ec2.internal" event="NodeReady"
Jul 19 14:13:36.021152 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.021097    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 19 14:13:36.038053 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.038030    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kxf6j_42db642d-3fb7-4e9c-a3f2-e726a97bf514/node-ca/0.log"
Jul 19 14:13:36.052797 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.052770    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f888656df-hq4mp"]
Jul 19 14:13:36.056959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.056931    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.060069 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.060033    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 19 14:13:36.060069 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.060045    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 19 14:13:36.060293 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.060272    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-knpqf\""
Jul 19 14:13:36.060507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.060492    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 19 14:13:36.065609 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.065587    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hqqjj"]
Jul 19 14:13:36.067235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.067214    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 19 14:13:36.069985 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.069964    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8795k"]
Jul 19 14:13:36.070106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.070090    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.072734 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.072556    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 19 14:13:36.072734 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.072570    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n5648\""
Jul 19 14:13:36.072734 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.072583    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 19 14:13:36.072925 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.072803    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 19 14:13:36.072925 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.072868    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 19 14:13:36.075016 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.074999    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f888656df-hq4mp"]
Jul 19 14:13:36.075113 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.075100    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.077482 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.077386    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 19 14:13:36.077939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.077920    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 19 14:13:36.078032 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.078008    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n7vcv\""
Jul 19 14:13:36.079109 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.078267    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hqqjj"]
Jul 19 14:13:36.082821 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.082719    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8795k"]
Jul 19 14:13:36.091060 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.091042    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wrm4x"]
Jul 19 14:13:36.095145 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.095124    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.097550 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.097517    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 19 14:13:36.097625 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.097579    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 19 14:13:36.097625 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.097596    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 19 14:13:36.097718 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.097517    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5dfvv\""
Jul 19 14:13:36.103266 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.103249    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wrm4x"]
Jul 19 14:13:36.187783 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.187748    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:36.188761 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.188738    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc3b04c5-6b23-40f6-b8a8-426845de5095-ca-trust-extracted\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.188884 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.188768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.188884 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.188790    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r59d\" (UniqueName: \"kubernetes.io/projected/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-api-access-7r59d\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.188884 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.188837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efc9cd0e-1194-4797-a2b8-2ef8092c1985-tmp-dir\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.188982 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.188921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-trusted-ca\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.189018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.188977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-bound-sa-token\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.189018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwjzb\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-kube-api-access-nwjzb\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.189091 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/187bc853-e9de-411a-a649-f239f8f3dcfc-crio-socket\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.189091 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189062    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qf5h\" (UniqueName: \"kubernetes.io/projected/665147b0-a844-4f47-a7db-38498a65ea47-kube-api-access-5qf5h\") pod \"ingress-canary-wrm4x\" (UID: \"665147b0-a844-4f47-a7db-38498a65ea47\") " pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.189183 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.189231 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-image-registry-private-configuration\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.189268 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.189305 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/665147b0-a844-4f47-a7db-38498a65ea47-cert\") pod \"ingress-canary-wrm4x\" (UID: \"665147b0-a844-4f47-a7db-38498a65ea47\") " pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.189335 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-installation-pull-secrets\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.189335 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189330    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/187bc853-e9de-411a-a649-f239f8f3dcfc-data-volume\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.189401 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efc9cd0e-1194-4797-a2b8-2ef8092c1985-config-volume\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.189446 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189401    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-certificates\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.189498 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8b9v\" (UniqueName: \"kubernetes.io/projected/efc9cd0e-1194-4797-a2b8-2ef8092c1985-kube-api-access-h8b9v\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.189498 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.189487    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.290069 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-image-registry-private-configuration\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.290069 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290177    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/665147b0-a844-4f47-a7db-38498a65ea47-cert\") pod \"ingress-canary-wrm4x\" (UID: \"665147b0-a844-4f47-a7db-38498a65ea47\") " pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290234    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls podName:efc9cd0e-1194-4797-a2b8-2ef8092c1985 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:36.790216337 +0000 UTC m=+35.093047333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls") pod "dns-default-8795k" (UID: "efc9cd0e-1194-4797-a2b8-2ef8092c1985") : secret "dns-default-metrics-tls" not found
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-installation-pull-secrets\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/187bc853-e9de-411a-a649-f239f8f3dcfc-data-volume\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efc9cd0e-1194-4797-a2b8-2ef8092c1985-config-volume\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.290340 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-certificates\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8b9v\" (UniqueName: \"kubernetes.io/projected/efc9cd0e-1194-4797-a2b8-2ef8092c1985-kube-api-access-h8b9v\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc3b04c5-6b23-40f6-b8a8-426845de5095-ca-trust-extracted\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290453    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290515    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290550    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f888656df-hq4mp: secret "image-registry-tls" not found
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290554    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290594    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls podName:cc3b04c5-6b23-40f6-b8a8-426845de5095 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:36.790578909 +0000 UTC m=+35.093409907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls") pod "image-registry-5f888656df-hq4mp" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095") : secret "image-registry-tls" not found
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.290613    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:36.790602483 +0000 UTC m=+35.093433483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r59d\" (UniqueName: \"kubernetes.io/projected/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-api-access-7r59d\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290668    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efc9cd0e-1194-4797-a2b8-2ef8092c1985-tmp-dir\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.290721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-trusted-ca\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-bound-sa-token\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwjzb\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-kube-api-access-nwjzb\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290798    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/187bc853-e9de-411a-a649-f239f8f3dcfc-crio-socket\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qf5h\" (UniqueName: \"kubernetes.io/projected/665147b0-a844-4f47-a7db-38498a65ea47-kube-api-access-5qf5h\") pod \"ingress-canary-wrm4x\" (UID: \"665147b0-a844-4f47-a7db-38498a65ea47\") " pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290866    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc3b04c5-6b23-40f6-b8a8-426845de5095-ca-trust-extracted\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290911    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efc9cd0e-1194-4797-a2b8-2ef8092c1985-config-volume\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.290919    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-certificates\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.291144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efc9cd0e-1194-4797-a2b8-2ef8092c1985-tmp-dir\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.291181    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/187bc853-e9de-411a-a649-f239f8f3dcfc-data-volume\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.291196    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:13:36.291299 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.291243    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:36.791230057 +0000 UTC m=+35.094061073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : configmap "kube-rbac-proxy" not found
Jul 19 14:13:36.291931 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.291455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/187bc853-e9de-411a-a649-f239f8f3dcfc-crio-socket\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.291931 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.291754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-trusted-ca\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.292727 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.292669    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:36.296121 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.295207    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-installation-pull-secrets\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.296121 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.295299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-image-registry-private-configuration\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.296918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.296900    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/665147b0-a844-4f47-a7db-38498a65ea47-cert\") pod \"ingress-canary-wrm4x\" (UID: \"665147b0-a844-4f47-a7db-38498a65ea47\") " pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.299358 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.299335    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8b9v\" (UniqueName: \"kubernetes.io/projected/efc9cd0e-1194-4797-a2b8-2ef8092c1985-kube-api-access-h8b9v\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.299944 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.299900    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwjzb\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-kube-api-access-nwjzb\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.300300 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.300283    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qf5h\" (UniqueName: \"kubernetes.io/projected/665147b0-a844-4f47-a7db-38498a65ea47-kube-api-access-5qf5h\") pod \"ingress-canary-wrm4x\" (UID: \"665147b0-a844-4f47-a7db-38498a65ea47\") " pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.300459 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.300437    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-bound-sa-token\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.300808 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.300788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r59d\" (UniqueName: \"kubernetes.io/projected/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-api-access-7r59d\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.390912 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.390828    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:13:36.391448 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.391011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:13:36.393484 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.393459    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:13:36.393484 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.393473    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:13:36.393670 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.393476    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7cp9\""
Jul 19 14:13:36.404378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.404359    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wrm4x"
Jul 19 14:13:36.608944 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.608908    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-hp97b"]
Jul 19 14:13:36.612484 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.612466    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.614920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.614901    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 19 14:13:36.615039 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.614935    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 19 14:13:36.615121 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.615052    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 19 14:13:36.615303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.615281    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 19 14:13:36.615427 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.615414    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-n5s68\""
Jul 19 14:13:36.618877 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.618600    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-hp97b"]
Jul 19 14:13:36.640122 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.640095    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:36.691972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.691898    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:36.694943 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.694917    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/46c062d4-f774-4ada-a379-2fdd8cc49895-signing-key\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.695079 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.694986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/46c062d4-f774-4ada-a379-2fdd8cc49895-signing-cabundle\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.695079 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.695033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfz49\" (UniqueName: \"kubernetes.io/projected/46c062d4-f774-4ada-a379-2fdd8cc49895-kube-api-access-qfz49\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.795736 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.795701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.795889 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.795744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz49\" (UniqueName: \"kubernetes.io/projected/46c062d4-f774-4ada-a379-2fdd8cc49895-kube-api-access-qfz49\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.795889 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.795789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:36.795889 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.795809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/46c062d4-f774-4ada-a379-2fdd8cc49895-signing-key\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.795889 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.795842    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:13:36.795889 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.795856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.795898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.795916    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:37.795895221 +0000 UTC m=+36.098726229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : configmap "kube-rbac-proxy" not found
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.795931    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.795978    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.796018    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:37.796002423 +0000 UTC m=+36.098833417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.796034    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls podName:efc9cd0e-1194-4797-a2b8-2ef8092c1985 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:37.796025601 +0000 UTC m=+36.098856600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls") pod "dns-default-8795k" (UID: "efc9cd0e-1194-4797-a2b8-2ef8092c1985") : secret "dns-default-metrics-tls" not found
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.796054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/46c062d4-f774-4ada-a379-2fdd8cc49895-signing-cabundle\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.796089    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:13:36.796138 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.796105    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f888656df-hq4mp: secret "image-registry-tls" not found
Jul 19 14:13:36.796558 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:36.796148    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls podName:cc3b04c5-6b23-40f6-b8a8-426845de5095 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:37.796133809 +0000 UTC m=+36.098964811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls") pod "image-registry-5f888656df-hq4mp" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095") : secret "image-registry-tls" not found
Jul 19 14:13:36.796712 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.796696    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/46c062d4-f774-4ada-a379-2fdd8cc49895-signing-cabundle\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.798347 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.798326    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/46c062d4-f774-4ada-a379-2fdd8cc49895-signing-key\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.803790 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.803767    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfz49\" (UniqueName: \"kubernetes.io/projected/46c062d4-f774-4ada-a379-2fdd8cc49895-kube-api-access-qfz49\") pod \"service-ca-58598695f7-hp97b\" (UID: \"46c062d4-f774-4ada-a379-2fdd8cc49895\") " pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:36.922579 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:36.922549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-hp97b"
Jul 19 14:13:37.391766 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.391727    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:13:37.394575 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.394471    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:13:37.394575 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.394495    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4hzhh\""
Jul 19 14:13:37.394575 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.394494    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:13:37.569512 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.569466    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:37.752945 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.752830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-hp97b"]
Jul 19 14:13:37.755978 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:37.755898    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c062d4_f774_4ada_a379_2fdd8cc49895.slice/crio-f3cf51cd9e7ee2dae4930b0b8c1cc00bcec21cf14da3317adfdf4e4c2c0d275b WatchSource:0}: Error finding container f3cf51cd9e7ee2dae4930b0b8c1cc00bcec21cf14da3317adfdf4e4c2c0d275b: Status 404 returned error can't find the container with id f3cf51cd9e7ee2dae4930b0b8c1cc00bcec21cf14da3317adfdf4e4c2c0d275b
Jul 19 14:13:37.756058 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.755977    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wrm4x"]
Jul 19 14:13:37.758027 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:37.758006    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod665147b0_a844_4f47_a7db_38498a65ea47.slice/crio-3b676ee1f5c90ab932eba3d8af644df457e9f603ec85987646246da3d617fa78 WatchSource:0}: Error finding container 3b676ee1f5c90ab932eba3d8af644df457e9f603ec85987646246da3d617fa78: Status 404 returned error can't find the container with id 3b676ee1f5c90ab932eba3d8af644df457e9f603ec85987646246da3d617fa78
Jul 19 14:13:37.804458 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.804423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:37.804592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.804478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:37.804592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.804548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:37.804592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:37.804581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804587    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804607    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f888656df-hq4mp: secret "image-registry-tls" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804646    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804660    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804652    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls podName:cc3b04c5-6b23-40f6-b8a8-426845de5095 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:39.804637594 +0000 UTC m=+38.107468588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls") pod "image-registry-5f888656df-hq4mp" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095") : secret "image-registry-tls" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804694    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804709    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:39.80469268 +0000 UTC m=+38.107523682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:37.804729 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804722    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls podName:efc9cd0e-1194-4797-a2b8-2ef8092c1985 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:39.804716649 +0000 UTC m=+38.107547643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls") pod "dns-default-8795k" (UID: "efc9cd0e-1194-4797-a2b8-2ef8092c1985") : secret "dns-default-metrics-tls" not found
Jul 19 14:13:37.804986 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:37.804743    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:39.804730042 +0000 UTC m=+38.107561037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : configmap "kube-rbac-proxy" not found
Jul 19 14:13:38.015148 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:38.015022    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:38.670251 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:38.670208    2574 generic.go:356] "Generic (PLEG): container finished" podID="8633cae0-26e7-4e0b-a312-084901853626" containerID="a1a874b6f32a860849338be0c2893935159e8aec075552c17a8110cfd2f4828d" exitCode=0
Jul 19 14:13:38.671000 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:38.670325    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerDied","Data":"a1a874b6f32a860849338be0c2893935159e8aec075552c17a8110cfd2f4828d"}
Jul 19 14:13:38.672093 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:38.672068    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-hp97b" event={"ID":"46c062d4-f774-4ada-a379-2fdd8cc49895","Type":"ContainerStarted","Data":"f3cf51cd9e7ee2dae4930b0b8c1cc00bcec21cf14da3317adfdf4e4c2c0d275b"}
Jul 19 14:13:38.673814 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:38.673778    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wrm4x" event={"ID":"665147b0-a844-4f47-a7db-38498a65ea47","Type":"ContainerStarted","Data":"3b676ee1f5c90ab932eba3d8af644df457e9f603ec85987646246da3d617fa78"}
Jul 19 14:13:39.681680 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.681639    2574 generic.go:356] "Generic (PLEG): container finished" podID="8633cae0-26e7-4e0b-a312-084901853626" containerID="f1a8fbda64180e0d9a4a7fd382e6ffe4f619cdd1ee40bc235e630ede581e39b3" exitCode=0
Jul 19 14:13:39.682102 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.681736    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerDied","Data":"f1a8fbda64180e0d9a4a7fd382e6ffe4f619cdd1ee40bc235e630ede581e39b3"}
Jul 19 14:13:39.820446 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.820413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:39.820642 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.820502    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:39.820642 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.820582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:39.820760 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.820653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:39.820760 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.820668    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:13:39.820760 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.820754    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls podName:efc9cd0e-1194-4797-a2b8-2ef8092c1985 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:43.820733828 +0000 UTC m=+42.123564843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls") pod "dns-default-8795k" (UID: "efc9cd0e-1194-4797-a2b8-2ef8092c1985") : secret "dns-default-metrics-tls" not found
Jul 19 14:13:39.820995 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.820962    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/187bc853-e9de-411a-a649-f239f8f3dcfc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:39.820995 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.820671    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:13:39.821127 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.821008    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f888656df-hq4mp: secret "image-registry-tls" not found
Jul 19 14:13:39.821127 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.821035    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:39.821127 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.821052    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls podName:cc3b04c5-6b23-40f6-b8a8-426845de5095 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:43.821039876 +0000 UTC m=+42.123870873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls") pod "image-registry-5f888656df-hq4mp" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095") : secret "image-registry-tls" not found
Jul 19 14:13:39.821127 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:39.821080    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:43.821063608 +0000 UTC m=+42.123894611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:39.884660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:39.884621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:41.689928 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.689773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" event={"ID":"8633cae0-26e7-4e0b-a312-084901853626","Type":"ContainerStarted","Data":"a729e5743d50005da8afb9c59a03deb7a2908b7860931ad9a67cff6d0afffa3f"}
Jul 19 14:13:41.691210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.691186    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-hp97b" event={"ID":"46c062d4-f774-4ada-a379-2fdd8cc49895","Type":"ContainerStarted","Data":"e43b9c7204fcfc02c6a229dd832e701f670359afe65baac205bda4304512b484"}
Jul 19 14:13:41.692569 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.692550    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wrm4x" event={"ID":"665147b0-a844-4f47-a7db-38498a65ea47","Type":"ContainerStarted","Data":"64ab3cb433209b50f985e870c126b4aabab7b64c1a4d4aec09332e85f048a4de"}
Jul 19 14:13:41.719707 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.719641    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ssfg4" podStartSLOduration=9.550192758 podStartE2EDuration="39.719620081s" totalImagesPullingTime="30.169427323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:41.718493649 +0000 UTC m=+40.021324667" watchObservedRunningTime="2026-07-19 14:13:41.719620081 +0000 UTC m=+40.022451100"
Jul 19 14:13:41.735626 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.735572    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wrm4x" podStartSLOduration=2.90089022 podStartE2EDuration="5.735552132s" totalImagesPullingTime="2.834661912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:41.734707427 +0000 UTC m=+40.037538438" watchObservedRunningTime="2026-07-19 14:13:41.735552132 +0000 UTC m=+40.038383146"
Jul 19 14:13:41.747818 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.747786    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:41.750821 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.750776    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-hp97b" podStartSLOduration=2.9189987569999998 podStartE2EDuration="5.75075705s" totalImagesPullingTime="2.831758293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:41.75001116 +0000 UTC m=+40.052842177" watchObservedRunningTime="2026-07-19 14:13:41.75075705 +0000 UTC m=+40.053588066"
Jul 19 14:13:41.806762 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:41.806729    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.105704 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.105621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.300343 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.300296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.352989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.352955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.415487 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.415448    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.516962 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.516925    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.620975 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.620935    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:42.802415 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:42.802319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:43.608855 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:43.608818    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:43.855146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:43.855113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:43.855146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:43.855158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:43.855219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855272    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855291    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f888656df-hq4mp: secret "image-registry-tls" not found
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855342    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls podName:cc3b04c5-6b23-40f6-b8a8-426845de5095 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:51.855324989 +0000 UTC m=+50.158155984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls") pod "image-registry-5f888656df-hq4mp" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095") : secret "image-registry-tls" not found
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855343    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855373    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855396    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls podName:187bc853-e9de-411a-a649-f239f8f3dcfc nodeName:}" failed. No retries permitted until 2026-07-19 14:13:51.855380104 +0000 UTC m=+50.158211098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqqjj" (UID: "187bc853-e9de-411a-a649-f239f8f3dcfc") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:13:43.855593 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:13:43.855425    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls podName:efc9cd0e-1194-4797-a2b8-2ef8092c1985 nodeName:}" failed. No retries permitted until 2026-07-19 14:13:51.855414133 +0000 UTC m=+50.158245130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls") pod "dns-default-8795k" (UID: "efc9cd0e-1194-4797-a2b8-2ef8092c1985") : secret "dns-default-metrics-tls" not found
Jul 19 14:13:43.926582 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:43.926501    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:44.026949 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:44.026914    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:44.075863 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:44.075837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:44.208985 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:44.208907    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:44.399417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:44.399391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:44.800785 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:44.800752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:45.001090 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:45.001058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:45.408433 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:45.408403    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:45.796144 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:45.796059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:45.912991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:45.912963    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:46.210794 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:46.210760    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:46.603456 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:46.603375    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:46.799897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:46.799861    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:46.854231 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:46.854163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:47.003360 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:47.003308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:47.332645 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:47.332614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:47.433582 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:47.433554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:47.998820 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:47.998786    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:48.187191 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:48.187155    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:48.234674 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:48.234643    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:48.610909 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:48.610878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:48.800398 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:48.800366    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:49.111325 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:49.111296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:49.211158 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:49.211125    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:49.399161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:49.399131    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:49.801286 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:49.801203    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:50.399971 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:50.399935    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:50.799350 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:50.799268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:51.000249 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.000220    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:51.605203 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.605167    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:51.921835 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.921803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:51.921997 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.921857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:51.921997 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.921896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:51.925696 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.925669    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efc9cd0e-1194-4797-a2b8-2ef8092c1985-metrics-tls\") pod \"dns-default-8795k\" (UID: \"efc9cd0e-1194-4797-a2b8-2ef8092c1985\") " pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:51.925811 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.925716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"image-registry-5f888656df-hq4mp\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") " pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:51.925811 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.925713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/187bc853-e9de-411a-a649-f239f8f3dcfc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqqjj\" (UID: \"187bc853-e9de-411a-a649-f239f8f3dcfc\") " pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:51.968533 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.968504    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:51.983669 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.983640    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hqqjj"
Jul 19 14:13:51.988759 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:51.988742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:52.176093 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.175988    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f888656df-hq4mp"]
Jul 19 14:13:52.180213 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:52.180187    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3b04c5_6b23_40f6_b8a8_426845de5095.slice/crio-26f9964dfded1b2143e1d430cb49038440ee5e8a623066f2b0aacbad13a44628 WatchSource:0}: Error finding container 26f9964dfded1b2143e1d430cb49038440ee5e8a623066f2b0aacbad13a44628: Status 404 returned error can't find the container with id 26f9964dfded1b2143e1d430cb49038440ee5e8a623066f2b0aacbad13a44628
Jul 19 14:13:52.193871 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.193849    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hqqjj"]
Jul 19 14:13:52.197198 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:52.197171    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod187bc853_e9de_411a_a649_f239f8f3dcfc.slice/crio-f7ebd6daddb21ebc4efc4194cdc867a168f57db2a3a31d476b23f2bcbe8d0d80 WatchSource:0}: Error finding container f7ebd6daddb21ebc4efc4194cdc867a168f57db2a3a31d476b23f2bcbe8d0d80: Status 404 returned error can't find the container with id f7ebd6daddb21ebc4efc4194cdc867a168f57db2a3a31d476b23f2bcbe8d0d80
Jul 19 14:13:52.207389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.207356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:52.217596 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.217566    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8795k"]
Jul 19 14:13:52.222486 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:52.222468    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc9cd0e_1194_4797_a2b8_2ef8092c1985.slice/crio-1162821a46168c17cf3ff40b2590f7d6ad14d19d3586976c06ecbd4dda0bf23a WatchSource:0}: Error finding container 1162821a46168c17cf3ff40b2590f7d6ad14d19d3586976c06ecbd4dda0bf23a: Status 404 returned error can't find the container with id 1162821a46168c17cf3ff40b2590f7d6ad14d19d3586976c06ecbd4dda0bf23a
Jul 19 14:13:52.400857 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.400805    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:52.612283 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.612252    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:52.721493 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.721456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f888656df-hq4mp" event={"ID":"cc3b04c5-6b23-40f6-b8a8-426845de5095","Type":"ContainerStarted","Data":"8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575"}
Jul 19 14:13:52.721689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.721517    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f888656df-hq4mp" event={"ID":"cc3b04c5-6b23-40f6-b8a8-426845de5095","Type":"ContainerStarted","Data":"26f9964dfded1b2143e1d430cb49038440ee5e8a623066f2b0aacbad13a44628"}
Jul 19 14:13:52.721689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.721566    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:13:52.722915 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.722845    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8795k" event={"ID":"efc9cd0e-1194-4797-a2b8-2ef8092c1985","Type":"ContainerStarted","Data":"1162821a46168c17cf3ff40b2590f7d6ad14d19d3586976c06ecbd4dda0bf23a"}
Jul 19 14:13:52.724507 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.724480    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqqjj" event={"ID":"187bc853-e9de-411a-a649-f239f8f3dcfc","Type":"ContainerStarted","Data":"f7ebd6daddb21ebc4efc4194cdc867a168f57db2a3a31d476b23f2bcbe8d0d80"}
Jul 19 14:13:52.740752 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.740701    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f888656df-hq4mp" podStartSLOduration=52.740683019 podStartE2EDuration="52.740683019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:52.739549942 +0000 UTC m=+51.042380958" watchObservedRunningTime="2026-07-19 14:13:52.740683019 +0000 UTC m=+51.043514034"
Jul 19 14:13:52.801806 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:52.801771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:54.719382 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.719353    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"]
Jul 19 14:13:54.756890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.756849    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"]
Jul 19 14:13:54.757019 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.756972    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.760058 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.760023    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 19 14:13:54.760058 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.760042    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 19 14:13:54.761175 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.761155    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 19 14:13:54.761305 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.761173    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 19 14:13:54.822613 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.822580    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f888656df-hq4mp"]
Jul 19 14:13:54.845778 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.845718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-tmp\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.845778 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.845778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.846028 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.845829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l8lv\" (UniqueName: \"kubernetes.io/projected/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-kube-api-access-6l8lv\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.914712 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.914671    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:54.930131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.930094    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dd8cfb444-cvsqf"]
Jul 19 14:13:54.946209 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.946177    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l8lv\" (UniqueName: \"kubernetes.io/projected/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-kube-api-access-6l8lv\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.946387 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.946289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-tmp\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.946387 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.946326    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.946703 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.946679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-tmp\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.949433 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.949406    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.960857 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.960776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l8lv\" (UniqueName: \"kubernetes.io/projected/d7e76751-2d8d-46ec-8dd7-a2113a2864ef-kube-api-access-6l8lv\") pod \"klusterlet-addon-workmgr-54cc554f77-rl9tv\" (UID: \"d7e76751-2d8d-46ec-8dd7-a2113a2864ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:54.988925 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.988846    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd8cfb444-cvsqf"]
Jul 19 14:13:54.989059 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.988976    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:54.993179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:54.993151    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:55.046972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.046944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-bound-sa-token\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.046980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wbrv\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-kube-api-access-7wbrv\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.047004    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb814f35-ecd0-427d-8b1c-e7851a01f734-image-registry-private-configuration\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.047032    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb814f35-ecd0-427d-8b1c-e7851a01f734-ca-trust-extracted\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.047082    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-registry-tls\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.047148    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb814f35-ecd0-427d-8b1c-e7851a01f734-registry-certificates\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.047173    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb814f35-ecd0-427d-8b1c-e7851a01f734-trusted-ca\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.047278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.047220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb814f35-ecd0-427d-8b1c-e7851a01f734-installation-pull-secrets\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.068005 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.067981    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:13:55.089400 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.089359    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:55.147794 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147762    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb814f35-ecd0-427d-8b1c-e7851a01f734-registry-certificates\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.147926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb814f35-ecd0-427d-8b1c-e7851a01f734-trusted-ca\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.147926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb814f35-ecd0-427d-8b1c-e7851a01f734-installation-pull-secrets\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.147926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-bound-sa-token\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.147926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147904    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrv\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-kube-api-access-7wbrv\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.147926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb814f35-ecd0-427d-8b1c-e7851a01f734-image-registry-private-configuration\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.148177 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.147947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb814f35-ecd0-427d-8b1c-e7851a01f734-ca-trust-extracted\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.148177 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.148132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-registry-tls\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.148566 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.148456    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb814f35-ecd0-427d-8b1c-e7851a01f734-ca-trust-extracted\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.148675 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.148612    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb814f35-ecd0-427d-8b1c-e7851a01f734-registry-certificates\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.148947 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.148925    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb814f35-ecd0-427d-8b1c-e7851a01f734-trusted-ca\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.150998 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.150764    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb814f35-ecd0-427d-8b1c-e7851a01f734-image-registry-private-configuration\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.150998 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.150880    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-registry-tls\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.150998 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.150945    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb814f35-ecd0-427d-8b1c-e7851a01f734-installation-pull-secrets\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.156594 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.156571    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-bound-sa-token\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.156729 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.156652    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wbrv\" (UniqueName: \"kubernetes.io/projected/fb814f35-ecd0-427d-8b1c-e7851a01f734-kube-api-access-7wbrv\") pod \"image-registry-6dd8cfb444-cvsqf\" (UID: \"fb814f35-ecd0-427d-8b1c-e7851a01f734\") " pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:55.189752 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.189721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:55.298849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:55.298772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:57.321571 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.321475    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"]
Jul 19 14:13:57.329446 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:57.329299    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e76751_2d8d_46ec_8dd7_a2113a2864ef.slice/crio-13e78edbbda33a65bdf1d70cb8a0e128f702934cc754bb40c19976499e484d2f WatchSource:0}: Error finding container 13e78edbbda33a65bdf1d70cb8a0e128f702934cc754bb40c19976499e484d2f: Status 404 returned error can't find the container with id 13e78edbbda33a65bdf1d70cb8a0e128f702934cc754bb40c19976499e484d2f
Jul 19 14:13:57.347233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.347205    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd8cfb444-cvsqf"]
Jul 19 14:13:57.352474 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:57.352454    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb814f35_ecd0_427d_8b1c_e7851a01f734.slice/crio-73efd065d801681f19316b0a48bd943bc365dd5551767da0a6a0e6985d47f98c WatchSource:0}: Error finding container 73efd065d801681f19316b0a48bd943bc365dd5551767da0a6a0e6985d47f98c: Status 404 returned error can't find the container with id 73efd065d801681f19316b0a48bd943bc365dd5551767da0a6a0e6985d47f98c
Jul 19 14:13:57.599420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.599346    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-zbb8r"]
Jul 19 14:13:57.602439 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.602414    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:13:57.603465 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.603441    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:57.604852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.604830    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nx8sh\""
Jul 19 14:13:57.604852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.604844    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 19 14:13:57.605010 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.604855    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 19 14:13:57.614389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.614352    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-zbb8r"]
Jul 19 14:13:57.665730 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.665693    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wszj2\" (UniqueName: \"kubernetes.io/projected/7447d118-fa0c-4fa0-829d-cc6f2ad4d67c-kube-api-access-wszj2\") pod \"downloads-5bc67dcd46-zbb8r\" (UID: \"7447d118-fa0c-4fa0-829d-cc6f2ad4d67c\") " pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:13:57.699471 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.699435    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:57.741352 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.741317    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8795k" event={"ID":"efc9cd0e-1194-4797-a2b8-2ef8092c1985","Type":"ContainerStarted","Data":"1d8cf4d4328206873045941309a804ba20495a35fea176e6460c416300109796"}
Jul 19 14:13:57.741546 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.741382    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8795k" event={"ID":"efc9cd0e-1194-4797-a2b8-2ef8092c1985","Type":"ContainerStarted","Data":"8f29ff22415b3a6d44754f03a084a7659d93f257b2adeeef090b99224be0734a"}
Jul 19 14:13:57.741546 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.741446    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8795k"
Jul 19 14:13:57.743213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.743179    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqqjj" event={"ID":"187bc853-e9de-411a-a649-f239f8f3dcfc","Type":"ContainerStarted","Data":"96ad0de5fa1090751d86434055c28b821d1d8f0308d60d2dea564874ef5ca588"}
Jul 19 14:13:57.745503 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.745476    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf" event={"ID":"fb814f35-ecd0-427d-8b1c-e7851a01f734","Type":"ContainerStarted","Data":"a6d87191ca1f7281e86f9429a4056480c8dcea0b477dbc9791c3c9f55ab61eec"}
Jul 19 14:13:57.745636 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.745541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf" event={"ID":"fb814f35-ecd0-427d-8b1c-e7851a01f734","Type":"ContainerStarted","Data":"73efd065d801681f19316b0a48bd943bc365dd5551767da0a6a0e6985d47f98c"}
Jul 19 14:13:57.745636 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.745569    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:13:57.746830 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.746795    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv" event={"ID":"d7e76751-2d8d-46ec-8dd7-a2113a2864ef","Type":"ContainerStarted","Data":"13e78edbbda33a65bdf1d70cb8a0e128f702934cc754bb40c19976499e484d2f"}
Jul 19 14:13:57.756950 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.756855    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8795k" podStartSLOduration=16.812534442 podStartE2EDuration="21.75683676s" totalImagesPullingTime="4.944302318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:57.756276575 +0000 UTC m=+56.059107734" watchObservedRunningTime="2026-07-19 14:13:57.75683676 +0000 UTC m=+56.059667777"
Jul 19 14:13:57.758264 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.758243    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:57.767200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.767135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wszj2\" (UniqueName: \"kubernetes.io/projected/7447d118-fa0c-4fa0-829d-cc6f2ad4d67c-kube-api-access-wszj2\") pod \"downloads-5bc67dcd46-zbb8r\" (UID: \"7447d118-fa0c-4fa0-829d-cc6f2ad4d67c\") " pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:13:57.775757 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.775713    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf" podStartSLOduration=3.775698865 podStartE2EDuration="3.775698865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:13:57.774582893 +0000 UTC m=+56.077413912" watchObservedRunningTime="2026-07-19 14:13:57.775698865 +0000 UTC m=+56.078529859"
Jul 19 14:13:57.778726 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.778674    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wszj2\" (UniqueName: \"kubernetes.io/projected/7447d118-fa0c-4fa0-829d-cc6f2ad4d67c-kube-api-access-wszj2\") pod \"downloads-5bc67dcd46-zbb8r\" (UID: \"7447d118-fa0c-4fa0-829d-cc6f2ad4d67c\") " pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:13:57.873240 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.873199    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:57.916265 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:57.916229    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:13:58.082381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.082307    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-zbb8r"]
Jul 19 14:13:58.091353 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:13:58.091324    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7447d118_fa0c_4fa0_829d_cc6f2ad4d67c.slice/crio-5992a8c549770b8b6e29f38b1ce8463afd79140648cef6147defd2ea6298fa00 WatchSource:0}: Error finding container 5992a8c549770b8b6e29f38b1ce8463afd79140648cef6147defd2ea6298fa00: Status 404 returned error can't find the container with id 5992a8c549770b8b6e29f38b1ce8463afd79140648cef6147defd2ea6298fa00
Jul 19 14:13:58.268154 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.268052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:58.321418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.321301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:58.682729 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.682685    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:58.753227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.753158    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-zbb8r" event={"ID":"7447d118-fa0c-4fa0-829d-cc6f2ad4d67c","Type":"ContainerStarted","Data":"5992a8c549770b8b6e29f38b1ce8463afd79140648cef6147defd2ea6298fa00"}
Jul 19 14:13:58.890418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.890351    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:58.941091 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:58.940980    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:59.077919 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:59.077884    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:59.130496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:59.130425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:59.272219 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:59.272087    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:13:59.525410 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:13:59.525293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:01.091391 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:01.091304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:01.143110 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:01.143060    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:01.666205 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:01.666176    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h4r72"
Jul 19 14:14:01.824306 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:01.824250    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:01.894569 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:01.894514    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:02.788474 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:02.788395    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:02.892267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:02.892241    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:02.989329 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:02.989292    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:03.090514 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.090374    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:03.166310 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.166268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:03.218791 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.218757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:03.339483 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.339451    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:03.773301 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.773206    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv" event={"ID":"d7e76751-2d8d-46ec-8dd7-a2113a2864ef","Type":"ContainerStarted","Data":"ce6566d6ea14cb2f5216f5060ab79091f09bcc68c1c874bff1f2d59acd5481d9"}
Jul 19 14:14:03.773585 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.773510    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:14:03.775242 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.775220    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv"
Jul 19 14:14:03.775508 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.775475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqqjj" event={"ID":"187bc853-e9de-411a-a649-f239f8f3dcfc","Type":"ContainerStarted","Data":"d02d5eb47ca5bfe08177b5ebce5becb82eb1033ddc8bc9ff1584bc3ab5e285e9"}
Jul 19 14:14:03.789345 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:03.789293    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54cc554f77-rl9tv" podStartSLOduration=3.692103824 podStartE2EDuration="9.789274339s" totalImagesPullingTime="6.097170515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:03.788195359 +0000 UTC m=+62.091026376" watchObservedRunningTime="2026-07-19 14:14:03.789274339 +0000 UTC m=+62.092105357"
Jul 19 14:14:04.258718 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:04.258675    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:04.339535 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:04.339485    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:04.439704 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:04.439659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:04.792657 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:04.792626    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:04.891199 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:04.891145    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:05.908021 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.907977    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c564d7dcb-58gc4"]
Jul 19 14:14:05.912849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.912824    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:05.915439 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.915417    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 19 14:14:05.916372 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.916349    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mm486\""
Jul 19 14:14:05.916466 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.916385    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 19 14:14:05.916566 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.916538    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 19 14:14:05.916633 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.916621    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 19 14:14:05.916729 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.916710    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 19 14:14:05.920848 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.920828    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c564d7dcb-58gc4"]
Jul 19 14:14:05.937174 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.937150    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-oauth-config\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:05.937306 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.937188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-console-config\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:05.937306 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.937223    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfkht\" (UniqueName: \"kubernetes.io/projected/de27c219-4f74-40c8-b8d5-837406db98d9-kube-api-access-zfkht\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:05.937418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.937313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-serving-cert\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:05.937418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.937342    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-oauth-serving-cert\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:05.937418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:05.937366    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-service-ca\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.038243 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.038208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-oauth-config\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.038420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.038253    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-console-config\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.038420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.038291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfkht\" (UniqueName: \"kubernetes.io/projected/de27c219-4f74-40c8-b8d5-837406db98d9-kube-api-access-zfkht\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.038420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.038321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-serving-cert\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.038420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.038353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-oauth-serving-cert\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.038420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.038383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-service-ca\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.039114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.039044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-service-ca\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.039114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.039049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-console-config\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.039539 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.039494    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-oauth-serving-cert\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.041077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.041056    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-oauth-config\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.042020 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.041999    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-serving-cert\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.047077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.047047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfkht\" (UniqueName: \"kubernetes.io/projected/de27c219-4f74-40c8-b8d5-837406db98d9-kube-api-access-zfkht\") pod \"console-6c564d7dcb-58gc4\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") " pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.226100 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.226016    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:06.946890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.946726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:14:06.946890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.946789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:14:06.949569 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.949408    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:14:06.949569 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.949423    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:14:06.959540 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.959506    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ced506fe-e66e-4a7f-9fed-f1c72377b3ec-original-pull-secret\") pod \"global-pull-secret-syncer-x7d24\" (UID: \"ced506fe-e66e-4a7f-9fed-f1c72377b3ec\") " pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:14:06.960026 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:06.959999    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7265185a-f739-4655-ad0c-0590debb071b-metrics-certs\") pod \"network-metrics-daemon-6kfzr\" (UID: \"7265185a-f739-4655-ad0c-0590debb071b\") " pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:14:07.004770 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.004730    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7cp9\""
Jul 19 14:14:07.007735 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.007706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7d24"
Jul 19 14:14:07.012839 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.012818    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kfzr"
Jul 19 14:14:07.047757 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.047733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:14:07.050260 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.050231    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:14:07.060570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.060550    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:14:07.070836 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.070809    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwg7j\" (UniqueName: \"kubernetes.io/projected/f8ae77e0-a2cd-4845-964e-a9709112c5a8-kube-api-access-gwg7j\") pod \"network-check-target-cdvtc\" (UID: \"f8ae77e0-a2cd-4845-964e-a9709112c5a8\") " pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:14:07.104264 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.104236    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4hzhh\""
Jul 19 14:14:07.111795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.111760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:14:07.468946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.468913    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6kfzr"]
Jul 19 14:14:07.473211 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:07.472949    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7265185a_f739_4655_ad0c_0590debb071b.slice/crio-0ede39d12f5a70118a9404cc21e311e98f5a133ca84faf827fad09686c737826 WatchSource:0}: Error finding container 0ede39d12f5a70118a9404cc21e311e98f5a133ca84faf827fad09686c737826: Status 404 returned error can't find the container with id 0ede39d12f5a70118a9404cc21e311e98f5a133ca84faf827fad09686c737826
Jul 19 14:14:07.483345 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.483271    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x7d24"]
Jul 19 14:14:07.502674 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.502644    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c564d7dcb-58gc4"]
Jul 19 14:14:07.505294 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:07.505262    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde27c219_4f74_40c8_b8d5_837406db98d9.slice/crio-6508fb3dfdaf2fe274be29acddb037023c86438a6139fb47b3e8fb20b160439f WatchSource:0}: Error finding container 6508fb3dfdaf2fe274be29acddb037023c86438a6139fb47b3e8fb20b160439f: Status 404 returned error can't find the container with id 6508fb3dfdaf2fe274be29acddb037023c86438a6139fb47b3e8fb20b160439f
Jul 19 14:14:07.520471 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.520437    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cdvtc"]
Jul 19 14:14:07.524375 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:07.524344    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ae77e0_a2cd_4845_964e_a9709112c5a8.slice/crio-2e9e547c29a894be153deda32af0adb79a12ece05e6dd7d02394d10c9db5ff66 WatchSource:0}: Error finding container 2e9e547c29a894be153deda32af0adb79a12ece05e6dd7d02394d10c9db5ff66: Status 404 returned error can't find the container with id 2e9e547c29a894be153deda32af0adb79a12ece05e6dd7d02394d10c9db5ff66
Jul 19 14:14:07.755450 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.755417    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8795k"
Jul 19 14:14:07.790537 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.790478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cdvtc" event={"ID":"f8ae77e0-a2cd-4845-964e-a9709112c5a8","Type":"ContainerStarted","Data":"2e9e547c29a894be153deda32af0adb79a12ece05e6dd7d02394d10c9db5ff66"}
Jul 19 14:14:07.792898 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.792825    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqqjj" event={"ID":"187bc853-e9de-411a-a649-f239f8f3dcfc","Type":"ContainerStarted","Data":"b750dd0db675216886958777727bd477a66677bead27aba29761ffe8fef79bb8"}
Jul 19 14:14:07.795451 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.795403    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c564d7dcb-58gc4" event={"ID":"de27c219-4f74-40c8-b8d5-837406db98d9","Type":"ContainerStarted","Data":"6508fb3dfdaf2fe274be29acddb037023c86438a6139fb47b3e8fb20b160439f"}
Jul 19 14:14:07.796702 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.796676    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x7d24" event={"ID":"ced506fe-e66e-4a7f-9fed-f1c72377b3ec","Type":"ContainerStarted","Data":"e5867c17733e943606a6c76edd2926843d8bda43eeed6dae9eea9e1490eaf5e2"}
Jul 19 14:14:07.797998 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.797938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6kfzr" event={"ID":"7265185a-f739-4655-ad0c-0590debb071b","Type":"ContainerStarted","Data":"0ede39d12f5a70118a9404cc21e311e98f5a133ca84faf827fad09686c737826"}
Jul 19 14:14:07.810563 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:07.810432    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hqqjj" podStartSLOduration=16.779042159 podStartE2EDuration="31.810416578s" totalImagesPullingTime="15.031374419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:07.810205084 +0000 UTC m=+66.113036100" watchObservedRunningTime="2026-07-19 14:14:07.810416578 +0000 UTC m=+66.113247596"
Jul 19 14:14:08.150828 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.150791    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:08.189764 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.188550    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"]
Jul 19 14:14:08.193231 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.191942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.197509 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.196948    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 19 14:14:08.197509 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.197184    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bq6sx\""
Jul 19 14:14:08.197509 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.197363    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 19 14:14:08.198655 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.198391    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 19 14:14:08.199505 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.199384    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 19 14:14:08.202596 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.202183    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 19 14:14:08.203975 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.203808    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"]
Jul 19 14:14:08.218203 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.218179    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:08.258785 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.258689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.258785 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.258754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.259018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.258794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5whlc\" (UniqueName: \"kubernetes.io/projected/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-kube-api-access-5whlc\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.259018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.258826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.360192    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.360253    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.360282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5whlc\" (UniqueName: \"kubernetes.io/projected/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-kube-api-access-5whlc\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.360309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:14:08.360452    2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:14:08.360516    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-tls podName:5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee nodeName:}" failed. No retries permitted until 2026-07-19 14:14:08.860494464 +0000 UTC m=+67.163325478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-z25rn" (UID: "5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee") : secret "prometheus-operator-tls" not found
Jul 19 14:14:08.361787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.361401    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.369977 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.369927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.370498 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.370452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5whlc\" (UniqueName: \"kubernetes.io/projected/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-kube-api-access-5whlc\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.844245 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.844154    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:08.866156 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.865394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:08.874206 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:08.874106    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-z25rn\" (UID: \"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:09.116951 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:09.116914    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"
Jul 19 14:14:09.306216 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:09.306024    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-z25rn"]
Jul 19 14:14:09.443307 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:09.443235    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5408a7b9_89d1_4e9e_ac91_2ed4ce6bfbee.slice/crio-ca80d45fd9f876a6ad73a16eb9cce337fd0fa7daf5c76a9e8d9fccb79348519c WatchSource:0}: Error finding container ca80d45fd9f876a6ad73a16eb9cce337fd0fa7daf5c76a9e8d9fccb79348519c: Status 404 returned error can't find the container with id ca80d45fd9f876a6ad73a16eb9cce337fd0fa7daf5c76a9e8d9fccb79348519c
Jul 19 14:14:09.809479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:09.809337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6kfzr" event={"ID":"7265185a-f739-4655-ad0c-0590debb071b","Type":"ContainerStarted","Data":"d9f367f6aa6d856d5054525c1830931ee05e4c1c16b94b585afae36c90ea3e86"}
Jul 19 14:14:09.810968 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:09.810909    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn" event={"ID":"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee","Type":"ContainerStarted","Data":"ca80d45fd9f876a6ad73a16eb9cce337fd0fa7daf5c76a9e8d9fccb79348519c"}
Jul 19 14:14:10.414776 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:10.414723    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:10.467664 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:10.467632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:10.744407 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:10.744319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:10.845570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:10.845534    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:10.897079 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:10.897037    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:12.292269 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:12.292170    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:12.358215 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:12.358179    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:13.150473 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:13.150441    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:13.250501 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:13.250466    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:14.829101 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:14.829072    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:14:15.498127 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:15.498093    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:15.598395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:15.598365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:16.880650 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:16.880605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:16.982989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:16.982948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:18.758228 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:18.758201    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dd8cfb444-cvsqf"
Jul 19 14:14:18.879630 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:18.879596    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:18.978395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:18.978361    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:19.850211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:19.850167    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f888656df-hq4mp" podUID="cc3b04c5-6b23-40f6-b8a8-426845de5095" containerName="registry" containerID="cri-o://8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575" gracePeriod=30
Jul 19 14:14:21.563803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.563762    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:21.630724 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.630692    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:21.687090 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.686980    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:14:21.773275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773120    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc3b04c5-6b23-40f6-b8a8-426845de5095-ca-trust-extracted\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773184    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-installation-pull-secrets\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773213    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773253    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-image-registry-private-configuration\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773545 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773309    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-bound-sa-token\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773545 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773344    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-certificates\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773545 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773383    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwjzb\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-kube-api-access-nwjzb\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.773545 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.773424    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-trusted-ca\") pod \"cc3b04c5-6b23-40f6-b8a8-426845de5095\" (UID: \"cc3b04c5-6b23-40f6-b8a8-426845de5095\") "
Jul 19 14:14:21.774289 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.774010    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:14:21.774516 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.774460    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:14:21.777658 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.777468    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:14:21.777841 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.777803    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:14:21.778867 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.778834    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:14:21.779461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.779430    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:14:21.780311 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.780275    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-kube-api-access-nwjzb" (OuterVolumeSpecName: "kube-api-access-nwjzb") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "kube-api-access-nwjzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:14:21.785413 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.785387    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc3b04c5-6b23-40f6-b8a8-426845de5095-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cc3b04c5-6b23-40f6-b8a8-426845de5095" (UID: "cc3b04c5-6b23-40f6-b8a8-426845de5095"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:14:21.859638 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.859602    2574 generic.go:356] "Generic (PLEG): container finished" podID="cc3b04c5-6b23-40f6-b8a8-426845de5095" containerID="8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575" exitCode=0
Jul 19 14:14:21.859825 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.859686    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f888656df-hq4mp"
Jul 19 14:14:21.859939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.859684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f888656df-hq4mp" event={"ID":"cc3b04c5-6b23-40f6-b8a8-426845de5095","Type":"ContainerDied","Data":"8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575"}
Jul 19 14:14:21.860068 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.860008    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f888656df-hq4mp" event={"ID":"cc3b04c5-6b23-40f6-b8a8-426845de5095","Type":"ContainerDied","Data":"26f9964dfded1b2143e1d430cb49038440ee5e8a623066f2b0aacbad13a44628"}
Jul 19 14:14:21.860068 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.860047    2574 scope.go:117] "RemoveContainer" containerID="8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575"
Jul 19 14:14:21.861602 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.861362    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cdvtc" event={"ID":"f8ae77e0-a2cd-4845-964e-a9709112c5a8","Type":"ContainerStarted","Data":"f53e24c597731dbd6d25f6ea0c4afcac4195b6eaa8890df487a052ffce0e58da"}
Jul 19 14:14:21.861602 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.861408    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:14:21.866590 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.864713    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-zbb8r" event={"ID":"7447d118-fa0c-4fa0-829d-cc6f2ad4d67c","Type":"ContainerStarted","Data":"06d6dbd82e4b3b9730203b83dcadbd0984972d077df3b68ebfd4e27d185cd87b"}
Jul 19 14:14:21.866590 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.865885    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:14:21.867346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.867059    2574 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-zbb8r container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" start-of-body=
Jul 19 14:14:21.867346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.867107    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-zbb8r" podUID="7447d118-fa0c-4fa0-829d-cc6f2ad4d67c" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused"
Jul 19 14:14:21.868516 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.868490    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c564d7dcb-58gc4" event={"ID":"de27c219-4f74-40c8-b8d5-837406db98d9","Type":"ContainerStarted","Data":"78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f"}
Jul 19 14:14:21.870122 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.870102    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x7d24" event={"ID":"ced506fe-e66e-4a7f-9fed-f1c72377b3ec","Type":"ContainerStarted","Data":"41e309ad7eef0e32b6f583cf55e957381bfd4db63574885f282f23c1909e1924"}
Jul 19 14:14:21.872076 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.872050    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6kfzr" event={"ID":"7265185a-f739-4655-ad0c-0590debb071b","Type":"ContainerStarted","Data":"3671d7cfc8d7255e0941f8785606773ee7c9d6ba4d6bc07d0e7f2e479f083f20"}
Jul 19 14:14:21.874106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874081    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-bound-sa-token\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874108    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-certificates\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874121    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwjzb\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-kube-api-access-nwjzb\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874133    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc3b04c5-6b23-40f6-b8a8-426845de5095-trusted-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874146    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc3b04c5-6b23-40f6-b8a8-426845de5095-ca-trust-extracted\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874159    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-installation-pull-secrets\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874187    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc3b04c5-6b23-40f6-b8a8-426845de5095-registry-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.874200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.874201    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc3b04c5-6b23-40f6-b8a8-426845de5095-image-registry-private-configuration\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:21.881147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.881098    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cdvtc" podStartSLOduration=65.771465729 podStartE2EDuration="1m19.881081847s" totalImagesPullingTime="14.109616118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:21.88051375 +0000 UTC m=+80.183344767" watchObservedRunningTime="2026-07-19 14:14:21.881081847 +0000 UTC m=+80.183912841"
Jul 19 14:14:21.899123 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.899069    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-zbb8r" podStartSLOduration=1.3603235759999999 podStartE2EDuration="24.89905112s" totalImagesPullingTime="23.538727544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:21.89788199 +0000 UTC m=+80.200713027" watchObservedRunningTime="2026-07-19 14:14:21.89905112 +0000 UTC m=+80.201882136"
Jul 19 14:14:21.913789 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.913737    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c564d7dcb-58gc4" podStartSLOduration=2.785022547 podStartE2EDuration="16.913722996s" totalImagesPullingTime="14.128700449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:21.913019737 +0000 UTC m=+80.215850752" watchObservedRunningTime="2026-07-19 14:14:21.913722996 +0000 UTC m=+80.216554012"
Jul 19 14:14:21.926987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.926945    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x7d24" podStartSLOduration=65.781194251 podStartE2EDuration="1m19.926932103s" totalImagesPullingTime="14.145737852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:21.926555833 +0000 UTC m=+80.229386849" watchObservedRunningTime="2026-07-19 14:14:21.926932103 +0000 UTC m=+80.229763118"
Jul 19 14:14:21.939357 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.939329    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f888656df-hq4mp"]
Jul 19 14:14:21.942476 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.942452    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f888656df-hq4mp"]
Jul 19 14:14:21.955737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:21.955686    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6kfzr" podStartSLOduration=77.958149492 podStartE2EDuration="1m19.95567313s" totalImagesPullingTime="1.997523638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:21.955165888 +0000 UTC m=+80.257996905" watchObservedRunningTime="2026-07-19 14:14:21.95567313 +0000 UTC m=+80.258504145"
Jul 19 14:14:22.015136 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.015111    2574 scope.go:117] "RemoveContainer" containerID="8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575"
Jul 19 14:14:22.015426 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:14:22.015399    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575\": container with ID starting with 8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575 not found: ID does not exist" containerID="8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575"
Jul 19 14:14:22.015485 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.015439    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575"} err="failed to get container status \"8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575\": rpc error: code = NotFound desc = could not find container \"8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575\": container with ID starting with 8f5d04b78f0f94798786faf5d1aa64d7c74251ee9f5fcfca5947731da60aa575 not found: ID does not exist"
Jul 19 14:14:22.043434 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.043409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:22.145251 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.145219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:22.321363 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.321327    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:22.396817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.396779    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc3b04c5-6b23-40f6-b8a8-426845de5095" path="/var/lib/kubelet/pods/cc3b04c5-6b23-40f6-b8a8-426845de5095/volumes"
Jul 19 14:14:22.878402 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.878359    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn" event={"ID":"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee","Type":"ContainerStarted","Data":"3049f61ff881fb4b411886ef163416b7e337fd1e520a466e73335c80f27089c0"}
Jul 19 14:14:22.878883 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.878436    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn" event={"ID":"5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee","Type":"ContainerStarted","Data":"d68c364466ea89c4cda4f9c9a54ab4056615515c87e1b03cdc8b0e586296193a"}
Jul 19 14:14:22.896646 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.896606    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-zbb8r"
Jul 19 14:14:22.896984 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:22.896927    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-z25rn" podStartSLOduration=2.344448459 podStartE2EDuration="14.896907076s" totalImagesPullingTime="12.552458617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:22.892974844 +0000 UTC m=+81.195805872" watchObservedRunningTime="2026-07-19 14:14:22.896907076 +0000 UTC m=+81.199738092"
Jul 19 14:14:24.271847 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.271729    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:24.423397 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.423358    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c564d7dcb-58gc4"]
Jul 19 14:14:24.523449 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.523364    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:24.588314 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.588280    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"]
Jul 19 14:14:24.588724 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.588701    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc3b04c5-6b23-40f6-b8a8-426845de5095" containerName="registry"
Jul 19 14:14:24.588724 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.588726    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc3b04c5-6b23-40f6-b8a8-426845de5095" containerName="registry"
Jul 19 14:14:24.588881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.588782    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc3b04c5-6b23-40f6-b8a8-426845de5095" containerName="registry"
Jul 19 14:14:24.609593 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.609561    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-78hlt"]
Jul 19 14:14:24.609775 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.609673    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.612213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.612187    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 19 14:14:24.612580 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.612561    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 19 14:14:24.613280 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.613262    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mlrnd\""
Jul 19 14:14:24.624197 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.624172    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"]
Jul 19 14:14:24.624324 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.624306    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.626729 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.626703    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 19 14:14:24.628493 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.628030    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 19 14:14:24.628493 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.628308    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nxzhg\""
Jul 19 14:14:24.628666 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.628562    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 19 14:14:24.629615 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.628775    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-78hlt"]
Jul 19 14:14:24.629615 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.629229    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bxbmz"]
Jul 19 14:14:24.640932 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.640905    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:24.643149 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.643125    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.645789 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.645765    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 19 14:14:24.645914 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.645811    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 19 14:14:24.646071 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.645977    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 19 14:14:24.646141 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.646128    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6d9j7\""
Jul 19 14:14:24.697616 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697580    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697638    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768c69be-5675-444e-887f-18be8288fdd6-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697667    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2s5\" (UniqueName: \"kubernetes.io/projected/9028259e-9088-4c2c-89ce-79a693a6dc61-kube-api-access-cs2s5\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/768c69be-5675-444e-887f-18be8288fdd6-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87sl\" (UniqueName: \"kubernetes.io/projected/768c69be-5675-444e-887f-18be8288fdd6-kube-api-access-p87sl\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697782    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697807    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9028259e-9088-4c2c-89ce-79a693a6dc61-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.697852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.698213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697880    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.698213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.697905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.721999 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.721964    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:24.798682 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.798593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.798682 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.798658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768c69be-5675-444e-887f-18be8288fdd6-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.798895 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.798779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2s5\" (UniqueName: \"kubernetes.io/projected/9028259e-9088-4c2c-89ce-79a693a6dc61-kube-api-access-cs2s5\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.798895 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.798812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-textfile\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.798994 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.798940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/768c69be-5675-444e-887f-18be8288fdd6-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.799084 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.798978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fde7a40-2455-4de1-afba-fb2cce517190-metrics-client-ca\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.810177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-tls\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811033    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/768c69be-5675-444e-887f-18be8288fdd6-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811125    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p87sl\" (UniqueName: \"kubernetes.io/projected/768c69be-5675-444e-887f-18be8288fdd6-kube-api-access-p87sl\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-sys\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811830    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9028259e-9088-4c2c-89ce-79a693a6dc61-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811880    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.811908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-root\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.812500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.812514    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9028259e-9088-4c2c-89ce-79a693a6dc61-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.812727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnj6g\" (UniqueName: \"kubernetes.io/projected/5fde7a40-2455-4de1-afba-fb2cce517190-kube-api-access-fnj6g\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.813007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-accelerators-collector-config\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.814500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.813058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.815210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.813453    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.815210 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:14:24.813767    2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 19 14:14:24.815210 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:14:24.813850    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-tls podName:9028259e-9088-4c2c-89ce-79a693a6dc61 nodeName:}" failed. No retries permitted until 2026-07-19 14:14:25.313829799 +0000 UTC m=+83.616660842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-zmq6v" (UID: "9028259e-9088-4c2c-89ce-79a693a6dc61") : secret "openshift-state-metrics-tls" not found
Jul 19 14:14:24.815210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.813928    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.815210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.813963    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-wtmp\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.815210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.814156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768c69be-5675-444e-887f-18be8288fdd6-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.819293 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.819269    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87sl\" (UniqueName: \"kubernetes.io/projected/768c69be-5675-444e-887f-18be8288fdd6-kube-api-access-p87sl\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.828967 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.828898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.830869 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.830587    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.831250 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.831172    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2s5\" (UniqueName: \"kubernetes.io/projected/9028259e-9088-4c2c-89ce-79a693a6dc61-kube-api-access-cs2s5\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:24.834614 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.834429    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:24.835129 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.835085    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/768c69be-5675-444e-887f-18be8288fdd6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-78hlt\" (UID: \"768c69be-5675-444e-887f-18be8288fdd6\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.914739 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-wtmp\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-textfile\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fde7a40-2455-4de1-afba-fb2cce517190-metrics-client-ca\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-tls\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914891    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-sys\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-root\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.914979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.914976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnj6g\" (UniqueName: \"kubernetes.io/projected/5fde7a40-2455-4de1-afba-fb2cce517190-kube-api-access-fnj6g\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.915316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.915005    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-accelerators-collector-config\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.915933 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.915449    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-sys\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.915933 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.915724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-accelerators-collector-config\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.915933 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.915774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-textfile\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.915933 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.915898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-wtmp\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.916261 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.916198    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fde7a40-2455-4de1-afba-fb2cce517190-metrics-client-ca\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.916425 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.916404    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5fde7a40-2455-4de1-afba-fb2cce517190-root\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.918761 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.918737    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.918761 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.918758    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5fde7a40-2455-4de1-afba-fb2cce517190-node-exporter-tls\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.924809 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.924789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnj6g\" (UniqueName: \"kubernetes.io/projected/5fde7a40-2455-4de1-afba-fb2cce517190-kube-api-access-fnj6g\") pod \"node-exporter-bxbmz\" (UID: \"5fde7a40-2455-4de1-afba-fb2cce517190\") " pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.928774 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.928714    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:24.937427 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.937405    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt"
Jul 19 14:14:24.954756 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:24.954728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bxbmz"
Jul 19 14:14:24.984284 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:24.984257    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fde7a40_2455_4de1_afba_fb2cce517190.slice/crio-620da1de270cff8bb59da8a5dfc724de2b43aa100bbdb0421c0359e6b9674ae3 WatchSource:0}: Error finding container 620da1de270cff8bb59da8a5dfc724de2b43aa100bbdb0421c0359e6b9674ae3: Status 404 returned error can't find the container with id 620da1de270cff8bb59da8a5dfc724de2b43aa100bbdb0421c0359e6b9674ae3
Jul 19 14:14:25.030148 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.030030    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:25.121571 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.121534    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-78hlt"]
Jul 19 14:14:25.126878 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:25.126845    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768c69be_5675_444e_887f_18be8288fdd6.slice/crio-29a2589bfb853ad9ceb4fb52832cc922727651556772f709d168714edfc10ec7 WatchSource:0}: Error finding container 29a2589bfb853ad9ceb4fb52832cc922727651556772f709d168714edfc10ec7: Status 404 returned error can't find the container with id 29a2589bfb853ad9ceb4fb52832cc922727651556772f709d168714edfc10ec7
Jul 19 14:14:25.318193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.318122    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:25.320975 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.320945    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9028259e-9088-4c2c-89ce-79a693a6dc61-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zmq6v\" (UID: \"9028259e-9088-4c2c-89ce-79a693a6dc61\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:25.330430 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.330398    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:25.453010 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.452961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:25.522167 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.522125    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"
Jul 19 14:14:25.540699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.540663    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:25.664361 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.664326    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:25.711660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.711576    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v"]
Jul 19 14:14:25.716280 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:25.715963    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9028259e_9088_4c2c_89ce_79a693a6dc61.slice/crio-d0af63d9b5ba0bf4fdb90f46830c5593664a2bc2cf68147dda1ba00624a07dfa WatchSource:0}: Error finding container d0af63d9b5ba0bf4fdb90f46830c5593664a2bc2cf68147dda1ba00624a07dfa: Status 404 returned error can't find the container with id d0af63d9b5ba0bf4fdb90f46830c5593664a2bc2cf68147dda1ba00624a07dfa
Jul 19 14:14:25.886618 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.886552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:25.901568 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.901422    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v" event={"ID":"9028259e-9088-4c2c-89ce-79a693a6dc61","Type":"ContainerStarted","Data":"cb6d475ec491c153a419423565f970ba8e033448188b731b8dad294e159cce1d"}
Jul 19 14:14:25.901568 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.901492    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v" event={"ID":"9028259e-9088-4c2c-89ce-79a693a6dc61","Type":"ContainerStarted","Data":"d0af63d9b5ba0bf4fdb90f46830c5593664a2bc2cf68147dda1ba00624a07dfa"}
Jul 19 14:14:25.903343 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.903263    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxbmz" event={"ID":"5fde7a40-2455-4de1-afba-fb2cce517190","Type":"ContainerStarted","Data":"620da1de270cff8bb59da8a5dfc724de2b43aa100bbdb0421c0359e6b9674ae3"}
Jul 19 14:14:25.905742 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.905649    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt" event={"ID":"768c69be-5675-444e-887f-18be8288fdd6","Type":"ContainerStarted","Data":"29a2589bfb853ad9ceb4fb52832cc922727651556772f709d168714edfc10ec7"}
Jul 19 14:14:25.985469 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:25.985407    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:26.086660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:26.086590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:26.187295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:26.187219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:26.226541 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:26.226489    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:26.912823 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:26.912781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v" event={"ID":"9028259e-9088-4c2c-89ce-79a693a6dc61","Type":"ContainerStarted","Data":"c28fa3efe8939875c70ea85827c70afecd202089b139dd8f7f5ecd61f3999806"}
Jul 19 14:14:27.469918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.469885    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:27.570719 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.570685    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:27.671635 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.671553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:27.776065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.776012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:27.873374 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.873277    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:27.921249 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.920287    2574 generic.go:356] "Generic (PLEG): container finished" podID="5fde7a40-2455-4de1-afba-fb2cce517190" containerID="c64153d5b002ef3f807d48ac4a9bb6278fa8df4e2f5f6a0dfaa93695adcb5351" exitCode=0
Jul 19 14:14:27.921249 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.920370    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxbmz" event={"ID":"5fde7a40-2455-4de1-afba-fb2cce517190","Type":"ContainerDied","Data":"c64153d5b002ef3f807d48ac4a9bb6278fa8df4e2f5f6a0dfaa93695adcb5351"}
Jul 19 14:14:27.924393 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.924084    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt" event={"ID":"768c69be-5675-444e-887f-18be8288fdd6","Type":"ContainerStarted","Data":"537ce3d0255af2b3430a1602f9255e4faf0b62914eac894e5b01f454550001dd"}
Jul 19 14:14:27.924393 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:27.924144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt" event={"ID":"768c69be-5675-444e-887f-18be8288fdd6","Type":"ContainerStarted","Data":"e5491d7bd52d012ca599c1a8a38b36121d69daf8d9230c7d3618a0a43ece86e9"}
Jul 19 14:14:28.876103 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.876068    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"]
Jul 19 14:14:28.893610 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.893572    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"]
Jul 19 14:14:28.893779 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.893724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:28.897098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.896915    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5dbx9\""
Jul 19 14:14:28.897098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.896936    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 19 14:14:28.897098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.896948    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2jh1eem9gmbgc\""
Jul 19 14:14:28.897098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.896992    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 19 14:14:28.897411 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.897241    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 19 14:14:28.897466 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.897418    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 19 14:14:28.930991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.930956    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxbmz" event={"ID":"5fde7a40-2455-4de1-afba-fb2cce517190","Type":"ContainerStarted","Data":"f24dfb93ffd9213763b99ccd6c899125f9eccb91bb71d3c2a93a151406a6832c"}
Jul 19 14:14:28.931460 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.931011    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxbmz" event={"ID":"5fde7a40-2455-4de1-afba-fb2cce517190","Type":"ContainerStarted","Data":"358f43efaef491a50b3513d7e28f1667408a4b3962ab0843348d7cd1216b1163"}
Jul 19 14:14:28.931460 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.931225    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:28.933437 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.933412    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt" event={"ID":"768c69be-5675-444e-887f-18be8288fdd6","Type":"ContainerStarted","Data":"d23c4033edcd0b72061a611392fe584c19b6dfb31c1784dab8a447c6495d7eed"}
Jul 19 14:14:28.936038 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.936010    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v" event={"ID":"9028259e-9088-4c2c-89ce-79a693a6dc61","Type":"ContainerStarted","Data":"3fa8a77fc47d256efadac77df3b23b715682f905decdd3c36f425a7acb7763d7"}
Jul 19 14:14:28.949329 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.949253    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bxbmz" podStartSLOduration=2.249159223 podStartE2EDuration="4.94923972s" totalImagesPullingTime="2.543919161s" totalInitContainerRuntime="156.161336ms" isStatefulPod=true podCreationTimestamp="2026-07-19 14:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:28.94772121 +0000 UTC m=+87.250560636" watchObservedRunningTime="2026-07-19 14:14:28.94923972 +0000 UTC m=+87.252070714"
Jul 19 14:14:28.964928 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.964862    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zmq6v" podStartSLOduration=2.308878314 podStartE2EDuration="4.964844944s" totalImagesPullingTime="2.65596663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:28.963808653 +0000 UTC m=+87.266639671" watchObservedRunningTime="2026-07-19 14:14:28.964844944 +0000 UTC m=+87.267675960"
Jul 19 14:14:28.981534 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:28.981471    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-78hlt" podStartSLOduration=2.5748383649999997 podStartE2EDuration="4.9814563s" totalImagesPullingTime="2.406617935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:28.980285621 +0000 UTC m=+87.283116637" watchObservedRunningTime="2026-07-19 14:14:28.9814563 +0000 UTC m=+87.284287314"
Jul 19 14:14:29.001760 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.001727    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:29.051394 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.050899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-secret-metrics-server-tls\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.051394 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.051346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr5cc\" (UniqueName: \"kubernetes.io/projected/19861593-6358-4110-8ae1-2e8d1454956a-kube-api-access-sr5cc\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.051394 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.051382    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19861593-6358-4110-8ae1-2e8d1454956a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.051737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.051510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/19861593-6358-4110-8ae1-2e8d1454956a-audit-log\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.051737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.051576    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-secret-metrics-server-client-certs\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.051737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.051608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-client-ca-bundle\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.051737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.051694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/19861593-6358-4110-8ae1-2e8d1454956a-metrics-server-audit-profiles\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152287 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/19861593-6358-4110-8ae1-2e8d1454956a-audit-log\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152287 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152292    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-secret-metrics-server-client-certs\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152410    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-client-ca-bundle\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152473    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/19861593-6358-4110-8ae1-2e8d1454956a-metrics-server-audit-profiles\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152557    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-secret-metrics-server-tls\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr5cc\" (UniqueName: \"kubernetes.io/projected/19861593-6358-4110-8ae1-2e8d1454956a-kube-api-access-sr5cc\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152699 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19861593-6358-4110-8ae1-2e8d1454956a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.152956 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.152826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/19861593-6358-4110-8ae1-2e8d1454956a-audit-log\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.153502 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.153471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/19861593-6358-4110-8ae1-2e8d1454956a-metrics-server-audit-profiles\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.154001 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.153973    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19861593-6358-4110-8ae1-2e8d1454956a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.155338 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.155310    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-client-ca-bundle\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.155433 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.155400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-secret-metrics-server-client-certs\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.156078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.156056    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/19861593-6358-4110-8ae1-2e8d1454956a-secret-metrics-server-tls\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.162935 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.162913    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr5cc\" (UniqueName: \"kubernetes.io/projected/19861593-6358-4110-8ae1-2e8d1454956a-kube-api-access-sr5cc\") pod \"metrics-server-67b86fb4c4-9xsn2\" (UID: \"19861593-6358-4110-8ae1-2e8d1454956a\") " pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.206006 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.205895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:29.370922 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.370854    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:29.411074 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.411037    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"]
Jul 19 14:14:29.415148 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:29.415114    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19861593_6358_4110_8ae1_2e8d1454956a.slice/crio-cd481655d89436e5a1b306d3fadbfc3097da5f21e6267f7ab49683ee98b13bbe WatchSource:0}: Error finding container cd481655d89436e5a1b306d3fadbfc3097da5f21e6267f7ab49683ee98b13bbe: Status 404 returned error can't find the container with id cd481655d89436e5a1b306d3fadbfc3097da5f21e6267f7ab49683ee98b13bbe
Jul 19 14:14:29.500889 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.500807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:29.604618 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.604584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:29.731185 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.731155    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:29.810110 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.809970    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:29.810896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.810872    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b88665b5f-thcfn"]
Jul 19 14:14:29.840738 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.840701    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b88665b5f-thcfn"]
Jul 19 14:14:29.840916 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.840757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.843390 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.843361    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 19 14:14:29.843390 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.843362    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8xnnn\""
Jul 19 14:14:29.843607 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.843432    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 19 14:14:29.843607 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.843467    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 19 14:14:29.843607 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.843360    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 19 14:14:29.843856 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.843836    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 19 14:14:29.849908 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.849713    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 19 14:14:29.942570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.942508    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2" event={"ID":"19861593-6358-4110-8ae1-2e8d1454956a","Type":"ContainerStarted","Data":"cd481655d89436e5a1b306d3fadbfc3097da5f21e6267f7ab49683ee98b13bbe"}
Jul 19 14:14:29.959013 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.958979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-federate-client-tls\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959194 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vwzq\" (UniqueName: \"kubernetes.io/projected/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-kube-api-access-6vwzq\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959194 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959104    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-telemeter-client-tls\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959194 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-secret-telemeter-client\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959194 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959161    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-serving-certs-ca-bundle\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959194 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959184    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959414 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:29.959414 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:29.959282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-metrics-client-ca\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060405 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-federate-client-tls\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060405 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vwzq\" (UniqueName: \"kubernetes.io/projected/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-kube-api-access-6vwzq\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060674 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060574    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-telemeter-client-tls\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060674 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-secret-telemeter-client\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-serving-certs-ca-bundle\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060714    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060866 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.060866 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.060858    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-metrics-client-ca\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.061590 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.061561    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-metrics-client-ca\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.061713 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.061663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-serving-certs-ca-bundle\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.061851 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.061803    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.063283 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.063263    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-federate-client-tls\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.063816 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.063797    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-telemeter-client-tls\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.064168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.064144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.064168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.064156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-secret-telemeter-client\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.068375 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.068356    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vwzq\" (UniqueName: \"kubernetes.io/projected/21106d58-e2cd-4037-b7ce-3d1c0d23d3ec-kube-api-access-6vwzq\") pod \"telemeter-client-7b88665b5f-thcfn\" (UID: \"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec\") " pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.153247 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.153212    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn"
Jul 19 14:14:30.335422 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.335174    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b88665b5f-thcfn"]
Jul 19 14:14:30.337775 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:30.337734    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21106d58_e2cd_4037_b7ce_3d1c0d23d3ec.slice/crio-1e0b6a18ea306034b4b595206c7844be65aeffb768982fbbafa5067c22018ca9 WatchSource:0}: Error finding container 1e0b6a18ea306034b4b595206c7844be65aeffb768982fbbafa5067c22018ca9: Status 404 returned error can't find the container with id 1e0b6a18ea306034b4b595206c7844be65aeffb768982fbbafa5067c22018ca9
Jul 19 14:14:30.772939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.772892    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:14:30.775901 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.775868    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:30.803544 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.800065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:14:30.803544 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.800233    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.809228    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-snkxb\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.809491    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.809723    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7u71373osjobr\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.809916    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.811734    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.811983    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.812201    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 19 14:14:30.812316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.812250    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 19 14:14:30.814315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.813280    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 19 14:14:30.814315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.813297    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 19 14:14:30.814315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.813470    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 19 14:14:30.814315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.813474    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 19 14:14:30.814315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.813610    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 19 14:14:30.815124 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.814978    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 19 14:14:30.816985 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.816950    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 19 14:14:30.818958 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.818772    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 19 14:14:30.839158 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.839124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:30.951307 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.951265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn" event={"ID":"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec","Type":"ContainerStarted","Data":"1e0b6a18ea306034b4b595206c7844be65aeffb768982fbbafa5067c22018ca9"}
Jul 19 14:14:30.970127 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970212    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970239    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970439    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970461    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970495    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pllgz\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-kube-api-access-pllgz\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970788    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:30.970910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:30.970812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.071797 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071706    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pllgz\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-kube-api-access-pllgz\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.071797 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071777    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071854    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071937    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.071982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.072008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.072031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.072060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072633 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.072085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072633 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.072122    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.072633 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.072597    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.073247 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073011    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.073247 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.074807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.074807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.074807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.074807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.074807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.074807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.073805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.075165 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.075085    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.075343 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.075312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.076367 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.076285    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.076367 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.076318    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.076846 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.076819    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.077488 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.077128    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.077488 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.077444    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.078147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.077649    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.078147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.078037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.078439 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.078415    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.079081 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.079058    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.079820 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.079749    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.080063 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.080041    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.080656 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.080607    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.080930 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.080891    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.083714 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.083682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pllgz\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-kube-api-access-pllgz\") pod \"prometheus-k8s-0\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.126757 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.126721    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:31.440193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.440158    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:14:31.443697 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:31.443669    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4ba354_07c7_4e42_9470_def5e69d0d4a.slice/crio-b81b9b96f138dea12b6a39858be6082311461aec72d09217f6a5ec0bba1c9154 WatchSource:0}: Error finding container b81b9b96f138dea12b6a39858be6082311461aec72d09217f6a5ec0bba1c9154: Status 404 returned error can't find the container with id b81b9b96f138dea12b6a39858be6082311461aec72d09217f6a5ec0bba1c9154
Jul 19 14:14:31.957757 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.957677    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"b81b9b96f138dea12b6a39858be6082311461aec72d09217f6a5ec0bba1c9154"}
Jul 19 14:14:31.959557 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.959510    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2" event={"ID":"19861593-6358-4110-8ae1-2e8d1454956a","Type":"ContainerStarted","Data":"7063530887d247f7c85b5450e7f9f4a5e7c4bd78992060f61927bf14ea9c345c"}
Jul 19 14:14:31.977059 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:31.977011    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2" podStartSLOduration=1.7542633090000002 podStartE2EDuration="3.976990513s" totalImagesPullingTime="2.222727204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:31.976085976 +0000 UTC m=+90.278916992" watchObservedRunningTime="2026-07-19 14:14:31.976990513 +0000 UTC m=+90.279821530"
Jul 19 14:14:33.187935 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.187897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:33.285200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.285172    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:33.969663 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.969628    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="4c8bae112ec12038ce94972793be010d5cba3c4a14fcdbc146fdb410d013fa9f" exitCode=0
Jul 19 14:14:33.969851 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.969718    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"4c8bae112ec12038ce94972793be010d5cba3c4a14fcdbc146fdb410d013fa9f"}
Jul 19 14:14:33.971949 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.971929    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn" event={"ID":"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec","Type":"ContainerStarted","Data":"ab568f477e933141eedee7ce7750c2dd2a043723c28d8c012a1751eecd3257ab"}
Jul 19 14:14:33.972050 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.971968    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn" event={"ID":"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec","Type":"ContainerStarted","Data":"430a96abe38bcd7f66c008b856c0f9de5a420f2b79b5301e0afc8e38154974b1"}
Jul 19 14:14:33.972050 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:33.971991    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn" event={"ID":"21106d58-e2cd-4037-b7ce-3d1c0d23d3ec","Type":"ContainerStarted","Data":"e49f5b1d5a133b13d17871974999f30b398043ea9bddf75fef6c0c2020e9988e"}
Jul 19 14:14:34.032098 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:34.031999    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b88665b5f-thcfn" podStartSLOduration=1.900036696 podStartE2EDuration="5.031984251s" totalImagesPullingTime="3.131947555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:34.030755683 +0000 UTC m=+92.333586699" watchObservedRunningTime="2026-07-19 14:14:34.031984251 +0000 UTC m=+92.334815267"
Jul 19 14:14:34.884231 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:34.884195    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797b98b9c8-fsx82"]
Jul 19 14:14:34.906420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:34.906386    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797b98b9c8-fsx82"]
Jul 19 14:14:34.906608 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:34.906515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:34.917041 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:34.917014    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013167    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-service-ca\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-serving-cert\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-config\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-trusted-ca-bundle\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013370    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5cb8\" (UniqueName: \"kubernetes.io/projected/ad566a8c-1502-4a85-b01c-74cc1f0438b9-kube-api-access-m5cb8\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-oauth-serving-cert\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.013551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.013426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-oauth-config\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.040879 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.040768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:35.114554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-serving-cert\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.114554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-config\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.114554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-trusted-ca-bundle\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.114554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5cb8\" (UniqueName: \"kubernetes.io/projected/ad566a8c-1502-4a85-b01c-74cc1f0438b9-kube-api-access-m5cb8\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.114554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-oauth-serving-cert\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.114554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114448    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-oauth-config\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.114988 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.114898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-service-ca\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.115392 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.115255    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-config\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.115392 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.115355    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-trusted-ca-bundle\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.118204 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.118176    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-oauth-serving-cert\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.118474 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.118445    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:35.118740 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.118719    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-service-ca\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.119454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.119434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-serving-cert\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.121034 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.121010    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-oauth-config\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.123508 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.123468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5cb8\" (UniqueName: \"kubernetes.io/projected/ad566a8c-1502-4a85-b01c-74cc1f0438b9-kube-api-access-m5cb8\") pod \"console-797b98b9c8-fsx82\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") " pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.221236 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.221147    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:35.404009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.403891    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797b98b9c8-fsx82"]
Jul 19 14:14:35.407119 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:14:35.407089    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad566a8c_1502_4a85_b01c_74cc1f0438b9.slice/crio-4437c00e73ab968c400d3678d9f0db05d73e847c6b0d4a2cee1d03a62cd1472a WatchSource:0}: Error finding container 4437c00e73ab968c400d3678d9f0db05d73e847c6b0d4a2cee1d03a62cd1472a: Status 404 returned error can't find the container with id 4437c00e73ab968c400d3678d9f0db05d73e847c6b0d4a2cee1d03a62cd1472a
Jul 19 14:14:35.984402 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.984361    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797b98b9c8-fsx82" event={"ID":"ad566a8c-1502-4a85-b01c-74cc1f0438b9","Type":"ContainerStarted","Data":"228e91e7e4fcdbb23eaab823382b88e6d78962b21eca36ab6bd8dc395930c454"}
Jul 19 14:14:35.984964 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:35.984420    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797b98b9c8-fsx82" event={"ID":"ad566a8c-1502-4a85-b01c-74cc1f0438b9","Type":"ContainerStarted","Data":"4437c00e73ab968c400d3678d9f0db05d73e847c6b0d4a2cee1d03a62cd1472a"}
Jul 19 14:14:36.007212 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:36.007157    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797b98b9c8-fsx82" podStartSLOduration=2.007138557 podStartE2EDuration="2.007138557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:36.006240461 +0000 UTC m=+94.309071477" watchObservedRunningTime="2026-07-19 14:14:36.007138557 +0000 UTC m=+94.309969574"
Jul 19 14:14:37.996540 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:37.996481    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"85ca866fc909a0d3a67244b2f279b653eec6545462af66752cbacc88844d13a8"}
Jul 19 14:14:37.996929 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:37.996586    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"71400cf35a48c64cddbcbafee1862127658b71987849a3c19813cb77ab627d6d"}
Jul 19 14:14:41.011885 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:41.011847    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"44c6d6560c84de0cc62ef3582f347b3ffbfa588c74e528392c369d8015ef549b"}
Jul 19 14:14:41.012267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:41.011979    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"5a949a338a5145c9f6058866db139479539c78c3a0eedb36dfc9c7b1d5bd8594"}
Jul 19 14:14:41.012267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:41.012025    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"5ffefb0df7ec1e6dd13852285be7eb38cfc0683e42f1113047febc9becfcde5c"}
Jul 19 14:14:41.012267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:41.012061    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerStarted","Data":"f3fa849e8327111ebe73d2d8fa546f1ca66a813c78fbb9224a48a1c00ed28df0"}
Jul 19 14:14:41.040168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:41.040117    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.003005702 podStartE2EDuration="11.040101084s" totalImagesPullingTime="8.015329012s" totalInitContainerRuntime="21.76637ms" isStatefulPod=false podCreationTimestamp="2026-07-19 14:14:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:14:41.037176258 +0000 UTC m=+99.340007274" watchObservedRunningTime="2026-07-19 14:14:41.040101084 +0000 UTC m=+99.342932099"
Jul 19 14:14:41.127094 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:41.127062    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:14:45.221832 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:45.221788    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:45.222291 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:45.221860    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:45.226637 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:45.226614    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:46.033623 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:46.033594    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:14:46.182111 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:46.182080    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:46.234841 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:46.234810    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:46.288073 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:46.287974    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:48.177972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:48.177934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:48.241641 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:48.241607    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:14:49.206730 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.206698    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:49.207131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.206792    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:14:49.448364 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.448307    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c564d7dcb-58gc4" podUID="de27c219-4f74-40c8-b8d5-837406db98d9" containerName="console" containerID="cri-o://78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f" gracePeriod=15
Jul 19 14:14:49.725159 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.725131    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:49.746974 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.746910    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-oauth-serving-cert\") pod \"de27c219-4f74-40c8-b8d5-837406db98d9\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") "
Jul 19 14:14:49.747107 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.746977    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-service-ca\") pod \"de27c219-4f74-40c8-b8d5-837406db98d9\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") "
Jul 19 14:14:49.747107 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747024    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-oauth-config\") pod \"de27c219-4f74-40c8-b8d5-837406db98d9\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") "
Jul 19 14:14:49.747107 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747050    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-serving-cert\") pod \"de27c219-4f74-40c8-b8d5-837406db98d9\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") "
Jul 19 14:14:49.747107 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747093    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-console-config\") pod \"de27c219-4f74-40c8-b8d5-837406db98d9\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") "
Jul 19 14:14:49.747323 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747121    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfkht\" (UniqueName: \"kubernetes.io/projected/de27c219-4f74-40c8-b8d5-837406db98d9-kube-api-access-zfkht\") pod \"de27c219-4f74-40c8-b8d5-837406db98d9\" (UID: \"de27c219-4f74-40c8-b8d5-837406db98d9\") "
Jul 19 14:14:49.747323 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747272    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "de27c219-4f74-40c8-b8d5-837406db98d9" (UID: "de27c219-4f74-40c8-b8d5-837406db98d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:14:49.747421 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747377    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-oauth-serving-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:49.747421 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747386    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "de27c219-4f74-40c8-b8d5-837406db98d9" (UID: "de27c219-4f74-40c8-b8d5-837406db98d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:14:49.747500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.747465    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-console-config" (OuterVolumeSpecName: "console-config") pod "de27c219-4f74-40c8-b8d5-837406db98d9" (UID: "de27c219-4f74-40c8-b8d5-837406db98d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:14:49.749430 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.749403    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "de27c219-4f74-40c8-b8d5-837406db98d9" (UID: "de27c219-4f74-40c8-b8d5-837406db98d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:14:49.749558 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.749450    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "de27c219-4f74-40c8-b8d5-837406db98d9" (UID: "de27c219-4f74-40c8-b8d5-837406db98d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:14:49.749558 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.749449    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de27c219-4f74-40c8-b8d5-837406db98d9-kube-api-access-zfkht" (OuterVolumeSpecName: "kube-api-access-zfkht") pod "de27c219-4f74-40c8-b8d5-837406db98d9" (UID: "de27c219-4f74-40c8-b8d5-837406db98d9"). InnerVolumeSpecName "kube-api-access-zfkht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:14:49.847987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.847950    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-oauth-config\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:49.847987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.847979    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de27c219-4f74-40c8-b8d5-837406db98d9-console-serving-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:49.847987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.847989    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-console-config\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:49.848200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.847999    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfkht\" (UniqueName: \"kubernetes.io/projected/de27c219-4f74-40c8-b8d5-837406db98d9-kube-api-access-zfkht\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:49.848200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:49.848008    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de27c219-4f74-40c8-b8d5-837406db98d9-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:14:50.045542 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.045448    2574 generic.go:356] "Generic (PLEG): container finished" podID="de27c219-4f74-40c8-b8d5-837406db98d9" containerID="78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f" exitCode=0
Jul 19 14:14:50.045542 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.045495    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c564d7dcb-58gc4"
Jul 19 14:14:50.045719 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.045553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c564d7dcb-58gc4" event={"ID":"de27c219-4f74-40c8-b8d5-837406db98d9","Type":"ContainerDied","Data":"78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f"}
Jul 19 14:14:50.045719 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.045608    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c564d7dcb-58gc4" event={"ID":"de27c219-4f74-40c8-b8d5-837406db98d9","Type":"ContainerDied","Data":"6508fb3dfdaf2fe274be29acddb037023c86438a6139fb47b3e8fb20b160439f"}
Jul 19 14:14:50.045719 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.045634    2574 scope.go:117] "RemoveContainer" containerID="78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f"
Jul 19 14:14:50.053750 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.053734    2574 scope.go:117] "RemoveContainer" containerID="78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f"
Jul 19 14:14:50.054020 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:14:50.054004    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f\": container with ID starting with 78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f not found: ID does not exist" containerID="78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f"
Jul 19 14:14:50.054085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.054025    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f"} err="failed to get container status \"78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f\": rpc error: code = NotFound desc = could not find container \"78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f\": container with ID starting with 78557a5cc681d31c76d9b932c0cda6e02cb071f554a79f4aee3e93c2fc74d53f not found: ID does not exist"
Jul 19 14:14:50.066207 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.066185    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c564d7dcb-58gc4"]
Jul 19 14:14:50.069702 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.069682    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c564d7dcb-58gc4"]
Jul 19 14:14:50.395789 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:50.395750    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de27c219-4f74-40c8-b8d5-837406db98d9" path="/var/lib/kubelet/pods/de27c219-4f74-40c8-b8d5-837406db98d9/volumes"
Jul 19 14:14:52.883288 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:14:52.883258    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cdvtc"
Jul 19 14:15:09.211461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:09.211427    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:15:09.215078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:09.215055    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67b86fb4c4-9xsn2"
Jul 19 14:15:12.866060 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:12.866027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:15:31.127200 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:31.127166    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:31.145310 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:31.145280    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:31.244002 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:31.243977    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:40.621657 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:40.621621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:15:40.673226 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:40.673200    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:15:49.125618 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.125573    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:15:49.126070 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.125855    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="prometheus" containerID="cri-o://71400cf35a48c64cddbcbafee1862127658b71987849a3c19813cb77ab627d6d" gracePeriod=600
Jul 19 14:15:49.126070 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.125905    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="thanos-sidecar" containerID="cri-o://f3fa849e8327111ebe73d2d8fa546f1ca66a813c78fbb9224a48a1c00ed28df0" gracePeriod=600
Jul 19 14:15:49.126070 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.125955    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="config-reloader" containerID="cri-o://85ca866fc909a0d3a67244b2f279b653eec6545462af66752cbacc88844d13a8" gracePeriod=600
Jul 19 14:15:49.126070 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.125918    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-web" containerID="cri-o://5ffefb0df7ec1e6dd13852285be7eb38cfc0683e42f1113047febc9becfcde5c" gracePeriod=600
Jul 19 14:15:49.126318 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.126102    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://44c6d6560c84de0cc62ef3582f347b3ffbfa588c74e528392c369d8015ef549b" gracePeriod=600
Jul 19 14:15:49.126318 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.126227    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy" containerID="cri-o://5a949a338a5145c9f6058866db139479539c78c3a0eedb36dfc9c7b1d5bd8594" gracePeriod=600
Jul 19 14:15:49.297454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297402    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="44c6d6560c84de0cc62ef3582f347b3ffbfa588c74e528392c369d8015ef549b" exitCode=0
Jul 19 14:15:49.297454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297433    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="5a949a338a5145c9f6058866db139479539c78c3a0eedb36dfc9c7b1d5bd8594" exitCode=0
Jul 19 14:15:49.297454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297442    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="5ffefb0df7ec1e6dd13852285be7eb38cfc0683e42f1113047febc9becfcde5c" exitCode=0
Jul 19 14:15:49.297454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297449    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="f3fa849e8327111ebe73d2d8fa546f1ca66a813c78fbb9224a48a1c00ed28df0" exitCode=0
Jul 19 14:15:49.297454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297454    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="85ca866fc909a0d3a67244b2f279b653eec6545462af66752cbacc88844d13a8" exitCode=0
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297462    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerID="71400cf35a48c64cddbcbafee1862127658b71987849a3c19813cb77ab627d6d" exitCode=0
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"44c6d6560c84de0cc62ef3582f347b3ffbfa588c74e528392c369d8015ef549b"}
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297579    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"5a949a338a5145c9f6058866db139479539c78c3a0eedb36dfc9c7b1d5bd8594"}
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297619    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"5ffefb0df7ec1e6dd13852285be7eb38cfc0683e42f1113047febc9becfcde5c"}
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297658    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"f3fa849e8327111ebe73d2d8fa546f1ca66a813c78fbb9224a48a1c00ed28df0"}
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297697    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"85ca866fc909a0d3a67244b2f279b653eec6545462af66752cbacc88844d13a8"}
Jul 19 14:15:49.297788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.297732    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"71400cf35a48c64cddbcbafee1862127658b71987849a3c19813cb77ab627d6d"}
Jul 19 14:15:49.375873 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.375818    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:49.438267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438113    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-kube-rbac-proxy\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438169    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-tls\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438198    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-serving-certs-ca-bundle\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438240    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-2\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438264    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438290    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-metrics-client-ca\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438313    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-web-config\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438340    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-thanos-prometheus-http-client-file\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438365    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-tls-assets\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438400    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-1\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438427    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-kubelet-serving-ca-bundle\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438450    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438495    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-db\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438535    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438565    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config-out\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438615    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pllgz\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-kube-api-access-pllgz\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.438660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438660    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-metrics-client-certs\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438700    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-trusted-ca-bundle\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438727    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-grpc-tls\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438768    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-0\") pod \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\" (UID: \"9c4ba354-07c7-4e42-9470-def5e69d0d4a\") "
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438831    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438845    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.438887    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.439068    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-metrics-client-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.439089    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.439227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.439112    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.441149 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.439881    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:15:49.441149 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.440192    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.441341 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.441292    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:15:49.441574 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.441549    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.441986 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.441874    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.441986 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.441890    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.441986 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.441889    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config-out" (OuterVolumeSpecName: "config-out") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:15:49.442174 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.442125    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-kube-api-access-pllgz" (OuterVolumeSpecName: "kube-api-access-pllgz") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "kube-api-access-pllgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:15:49.442933 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.442906    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.443343 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.443306    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:15:49.444409 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.444309    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.444409 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.444329    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.444409 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.444371    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config" (OuterVolumeSpecName: "config") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.444689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.444580    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.444689 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.444615    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.445313 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.445281    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.453313 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.453290    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-web-config" (OuterVolumeSpecName: "web-config") pod "9c4ba354-07c7-4e42-9470-def5e69d0d4a" (UID: "9c4ba354-07c7-4e42-9470-def5e69d0d4a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:15:49.540196 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540167    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540196 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540193    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540196 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540203    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540213    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-web-config\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540221    2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540231    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-tls-assets\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540240    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540249    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-db\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540257    2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540264    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c4ba354-07c7-4e42-9470-def5e69d0d4a-config-out\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540273    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pllgz\" (UniqueName: \"kubernetes.io/projected/9c4ba354-07c7-4e42-9470-def5e69d0d4a-kube-api-access-pllgz\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540281    2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-metrics-client-certs\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540290    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540297    2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-grpc-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540305    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c4ba354-07c7-4e42-9470-def5e69d0d4a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540313    2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-kube-rbac-proxy\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:49.540395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:49.540321    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c4ba354-07c7-4e42-9470-def5e69d0d4a-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:15:50.306849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.306814    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c4ba354-07c7-4e42-9470-def5e69d0d4a","Type":"ContainerDied","Data":"b81b9b96f138dea12b6a39858be6082311461aec72d09217f6a5ec0bba1c9154"}
Jul 19 14:15:50.306849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.306846    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.307244 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.306896    2574 scope.go:117] "RemoveContainer" containerID="44c6d6560c84de0cc62ef3582f347b3ffbfa588c74e528392c369d8015ef549b"
Jul 19 14:15:50.315403 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.315381    2574 scope.go:117] "RemoveContainer" containerID="5a949a338a5145c9f6058866db139479539c78c3a0eedb36dfc9c7b1d5bd8594"
Jul 19 14:15:50.321814 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.321798    2574 scope.go:117] "RemoveContainer" containerID="5ffefb0df7ec1e6dd13852285be7eb38cfc0683e42f1113047febc9becfcde5c"
Jul 19 14:15:50.327981 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.327963    2574 scope.go:117] "RemoveContainer" containerID="f3fa849e8327111ebe73d2d8fa546f1ca66a813c78fbb9224a48a1c00ed28df0"
Jul 19 14:15:50.330275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.330251    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:15:50.334878 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.334860    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:15:50.335573 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.335556    2574 scope.go:117] "RemoveContainer" containerID="85ca866fc909a0d3a67244b2f279b653eec6545462af66752cbacc88844d13a8"
Jul 19 14:15:50.342005 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.341975    2574 scope.go:117] "RemoveContainer" containerID="71400cf35a48c64cddbcbafee1862127658b71987849a3c19813cb77ab627d6d"
Jul 19 14:15:50.348741 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.348724    2574 scope.go:117] "RemoveContainer" containerID="4c8bae112ec12038ce94972793be010d5cba3c4a14fcdbc146fdb410d013fa9f"
Jul 19 14:15:50.357996 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.357977    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:15:50.358258 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358246    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="thanos-sidecar"
Jul 19 14:15:50.358295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358259    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="thanos-sidecar"
Jul 19 14:15:50.358295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358268    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de27c219-4f74-40c8-b8d5-837406db98d9" containerName="console"
Jul 19 14:15:50.358295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358274    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de27c219-4f74-40c8-b8d5-837406db98d9" containerName="console"
Jul 19 14:15:50.358295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358288    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-thanos"
Jul 19 14:15:50.358295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358294    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-thanos"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358301    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-web"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358307    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-web"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358317    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358322    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358328    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="init-config-reloader"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358333    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="init-config-reloader"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358338    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="prometheus"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358343    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="prometheus"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358348    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="config-reloader"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358353    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="config-reloader"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358392    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-thanos"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358400    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de27c219-4f74-40c8-b8d5-837406db98d9" containerName="console"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358407    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="prometheus"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358413    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy-web"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358419    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="config-reloader"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358425    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="thanos-sidecar"
Jul 19 14:15:50.358428 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.358431    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" containerName="kube-rbac-proxy"
Jul 19 14:15:50.361796 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.361782    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.364389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364345    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 19 14:15:50.364649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364386    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 19 14:15:50.364649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364440    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-snkxb\""
Jul 19 14:15:50.364649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364352    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 19 14:15:50.364649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364484    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 19 14:15:50.364847 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364781    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 19 14:15:50.364980 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.364965    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 19 14:15:50.365035 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365021    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 19 14:15:50.365125 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365113    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 19 14:15:50.365321 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365307    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 19 14:15:50.365387 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365334    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 19 14:15:50.365387 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365373    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 19 14:15:50.365478 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365436    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7u71373osjobr\""
Jul 19 14:15:50.365643 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.365622    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 19 14:15:50.371552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.370202    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 19 14:15:50.376705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.376687    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 19 14:15:50.379649 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.379631    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:15:50.396097 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.396040    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c4ba354-07c7-4e42-9470-def5e69d0d4a" path="/var/lib/kubelet/pods/9c4ba354-07c7-4e42-9470-def5e69d0d4a/volumes"
Jul 19 14:15:50.445612 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445612 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445674    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxlff\" (UniqueName: \"kubernetes.io/projected/530ce3f8-0b31-46e0-a4e2-f27ac320555d-kube-api-access-hxlff\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445707    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-config\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.445803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/530ce3f8-0b31-46e0-a4e2-f27ac320555d-config-out\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445959    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.445981    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446085    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/530ce3f8-0b31-46e0-a4e2-f27ac320555d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-web-config\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.446239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.446235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547671 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547671 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547759    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxlff\" (UniqueName: \"kubernetes.io/projected/530ce3f8-0b31-46e0-a4e2-f27ac320555d-kube-api-access-hxlff\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-config\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/530ce3f8-0b31-46e0-a4e2-f27ac320555d-config-out\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.547920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547921    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.547984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548056    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/530ce3f8-0b31-46e0-a4e2-f27ac320555d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-web-config\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548122    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548326 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548765 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.548872 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.548850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.549124 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.549099    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.549807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.549787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.550767 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.550645    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/530ce3f8-0b31-46e0-a4e2-f27ac320555d-config-out\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.550941 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.550882    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551005 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.550974    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-web-config\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.551061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551478 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.551458    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551640 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.551489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-config\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551796 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.551774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551796 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.551789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.551956 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.551932    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.552415 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.552374    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/530ce3f8-0b31-46e0-a4e2-f27ac320555d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.553261 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.553233    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.553562 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.553545    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.553781 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.553758    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.553899 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.553883    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/530ce3f8-0b31-46e0-a4e2-f27ac320555d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.554435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.554415    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/530ce3f8-0b31-46e0-a4e2-f27ac320555d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.555624 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.555607    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxlff\" (UniqueName: \"kubernetes.io/projected/530ce3f8-0b31-46e0-a4e2-f27ac320555d-kube-api-access-hxlff\") pod \"prometheus-k8s-0\" (UID: \"530ce3f8-0b31-46e0-a4e2-f27ac320555d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.675494 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.675400    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:50.807605 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:50.807566    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:15:50.810967 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:15:50.810942    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530ce3f8_0b31_46e0_a4e2_f27ac320555d.slice/crio-928cd153edb6efe79dc8e96902517866df1014d59ac229337b5835d516791c0d WatchSource:0}: Error finding container 928cd153edb6efe79dc8e96902517866df1014d59ac229337b5835d516791c0d: Status 404 returned error can't find the container with id 928cd153edb6efe79dc8e96902517866df1014d59ac229337b5835d516791c0d
Jul 19 14:15:51.312043 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:51.312009    2574 generic.go:356] "Generic (PLEG): container finished" podID="530ce3f8-0b31-46e0-a4e2-f27ac320555d" containerID="d63cc74b0b9958a8253dc64392ab826c113a8a3302c7b3dceff71a7e2717964f" exitCode=0
Jul 19 14:15:51.312438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:51.312087    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerDied","Data":"d63cc74b0b9958a8253dc64392ab826c113a8a3302c7b3dceff71a7e2717964f"}
Jul 19 14:15:51.312438 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:51.312156    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"928cd153edb6efe79dc8e96902517866df1014d59ac229337b5835d516791c0d"}
Jul 19 14:15:52.319773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.319741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"57d822bb89e20d3bb0aea270b59919bb6d0ce156abe87af7a635d7186c1d13a3"}
Jul 19 14:15:52.320146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.319804    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"4e8ce293595211dd21a1271a315b339bf69bcae3eb9e7a22009acbad77dbd76c"}
Jul 19 14:15:52.320146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.319829    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"555c047b83c8a6d23f17a2b36b79968e8c2234f33194e5fd86c3a64161ffce8b"}
Jul 19 14:15:52.320146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.319852    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"9575b7971443b72889faf1f98f769b36939f2a98d1f832a872723f4debd344fd"}
Jul 19 14:15:52.320146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.319874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"8b764427b48f014f0bd860d24fbfb6a8b89d049cbb26448c09be7cfbd72f2f98"}
Jul 19 14:15:52.320146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.319896    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"530ce3f8-0b31-46e0-a4e2-f27ac320555d","Type":"ContainerStarted","Data":"88e7eb8ded42aa260f89d67d7176bb33060bd6c3eed1d49bb9e6ffe8e5d0cdc8"}
Jul 19 14:15:52.347933 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.347870    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.329306473 podStartE2EDuration="2.347852753s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.54628ms" isStatefulPod=false podCreationTimestamp="2026-07-19 14:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:15:52.34523169 +0000 UTC m=+170.648062707" watchObservedRunningTime="2026-07-19 14:15:52.347852753 +0000 UTC m=+170.650683768"
Jul 19 14:15:52.519260 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.519224    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:15:52.568690 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:52.568653    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:15:55.675637 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:55.675538    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:15:57.686773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:15:57.686737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:03.182168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:03.182133    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:03.282502 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:03.282472    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:03.330515 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:03.330476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:08.245865 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:08.245830    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797b98b9c8-fsx82"]
Jul 19 14:16:08.341725 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:08.341690    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:08.442365 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:08.442336    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:08.493345 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:08.493309    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:09.194855 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:09.194822    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:24.744733 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:24.744684    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:24.845334 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:24.845303    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:33.265330 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.265279    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-797b98b9c8-fsx82" podUID="ad566a8c-1502-4a85-b01c-74cc1f0438b9" containerName="console" containerID="cri-o://228e91e7e4fcdbb23eaab823382b88e6d78962b21eca36ab6bd8dc395930c454" gracePeriod=15
Jul 19 14:16:33.465113 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.465075    2574 generic.go:356] "Generic (PLEG): container finished" podID="ad566a8c-1502-4a85-b01c-74cc1f0438b9" containerID="228e91e7e4fcdbb23eaab823382b88e6d78962b21eca36ab6bd8dc395930c454" exitCode=0
Jul 19 14:16:33.465255 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.465141    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797b98b9c8-fsx82" event={"ID":"ad566a8c-1502-4a85-b01c-74cc1f0438b9","Type":"ContainerDied","Data":"228e91e7e4fcdbb23eaab823382b88e6d78962b21eca36ab6bd8dc395930c454"}
Jul 19 14:16:33.504217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.504196    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:16:33.609256 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609230    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-service-ca\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609272    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-trusted-ca-bundle\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609303    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-serving-cert\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609334    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-oauth-config\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609358    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-oauth-serving-cert\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609617 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609400    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-config\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609617 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609432    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5cb8\" (UniqueName: \"kubernetes.io/projected/ad566a8c-1502-4a85-b01c-74cc1f0438b9-kube-api-access-m5cb8\") pod \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\" (UID: \"ad566a8c-1502-4a85-b01c-74cc1f0438b9\") "
Jul 19 14:16:33.609709 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609684    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:16:33.609766 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609699    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-service-ca" (OuterVolumeSpecName: "service-ca") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:16:33.609972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609939    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-config" (OuterVolumeSpecName: "console-config") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:16:33.609972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.609954    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:16:33.611481 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.611454    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:16:33.611681 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.611650    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:16:33.611771 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.611732    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad566a8c-1502-4a85-b01c-74cc1f0438b9-kube-api-access-m5cb8" (OuterVolumeSpecName: "kube-api-access-m5cb8") pod "ad566a8c-1502-4a85-b01c-74cc1f0438b9" (UID: "ad566a8c-1502-4a85-b01c-74cc1f0438b9"). InnerVolumeSpecName "kube-api-access-m5cb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:16:33.710564 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710510    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:33.710564 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710559    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-trusted-ca-bundle\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:33.710564 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710569    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-serving-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:33.710787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710579    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-oauth-config\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:33.710787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710588    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-oauth-serving-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:33.710787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710597    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad566a8c-1502-4a85-b01c-74cc1f0438b9-console-config\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:33.710787 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:33.710606    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5cb8\" (UniqueName: \"kubernetes.io/projected/ad566a8c-1502-4a85-b01c-74cc1f0438b9-kube-api-access-m5cb8\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:16:34.470201 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:34.470171    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797b98b9c8-fsx82" event={"ID":"ad566a8c-1502-4a85-b01c-74cc1f0438b9","Type":"ContainerDied","Data":"4437c00e73ab968c400d3678d9f0db05d73e847c6b0d4a2cee1d03a62cd1472a"}
Jul 19 14:16:34.470616 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:34.470231    2574 scope.go:117] "RemoveContainer" containerID="228e91e7e4fcdbb23eaab823382b88e6d78962b21eca36ab6bd8dc395930c454"
Jul 19 14:16:34.470616 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:34.470186    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797b98b9c8-fsx82"
Jul 19 14:16:34.486282 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:34.486260    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797b98b9c8-fsx82"]
Jul 19 14:16:34.489303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:34.489285    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-797b98b9c8-fsx82"]
Jul 19 14:16:36.395515 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:36.395482    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad566a8c-1502-4a85-b01c-74cc1f0438b9" path="/var/lib/kubelet/pods/ad566a8c-1502-4a85-b01c-74cc1f0438b9/volumes"
Jul 19 14:16:38.792629 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:38.792597    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:38.892165 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:38.892135    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:16:50.676212 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:50.676180    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:16:50.690802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:50.690774    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:16:51.546078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:16:51.546052    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:17:05.430605 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:05.430574    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:06.213239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:06.213210    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:09.018011 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:09.017979    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:09.070386 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:09.070352    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:10.772950 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:10.772916    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:10.878362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:10.878333    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:11.102609 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:11.102542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:11.205959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:11.205930    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:13.497208 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:13.497173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:13.702262 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:13.702225    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:15.812706 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:15.812673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:15.872870 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:15.872819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:21.299363 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:21.299324    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:24.324067 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:24.323992    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:24.424844 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:24.424810    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:33.254876 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:33.254839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:33.350327 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:33.350284    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:39.247246 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:39.247193    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:39.348383 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:39.348347    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:41.179837 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:41.179805    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:53.469026 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.468987    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:53.513728 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.513703    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"]
Jul 19 14:17:53.514028 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.514015    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad566a8c-1502-4a85-b01c-74cc1f0438b9" containerName="console"
Jul 19 14:17:53.514073 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.514029    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad566a8c-1502-4a85-b01c-74cc1f0438b9" containerName="console"
Jul 19 14:17:53.514105 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.514095    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad566a8c-1502-4a85-b01c-74cc1f0438b9" containerName="console"
Jul 19 14:17:53.517357 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.517341    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.520865 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.520830    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 19 14:17:53.520865 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.520844    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 19 14:17:53.520865 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.520843    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 19 14:17:53.521085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.520898    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:17:53.521085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.521057    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-trktb\""
Jul 19 14:17:53.521165 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.521147    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 19 14:17:53.528679 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.528660    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"]
Jul 19 14:17:53.666401 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.666376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a93d13a3-027d-4269-b12d-37859e1b87cc-metrics-cert\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.666401 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.666404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lzrx\" (UniqueName: \"kubernetes.io/projected/a93d13a3-027d-4269-b12d-37859e1b87cc-kube-api-access-7lzrx\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.666617 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.666465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a93d13a3-027d-4269-b12d-37859e1b87cc-cert\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.666617 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.666517    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a93d13a3-027d-4269-b12d-37859e1b87cc-manager-config\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.767097 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.767040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a93d13a3-027d-4269-b12d-37859e1b87cc-cert\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.767097 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.767083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a93d13a3-027d-4269-b12d-37859e1b87cc-manager-config\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.767259 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.767111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a93d13a3-027d-4269-b12d-37859e1b87cc-metrics-cert\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.767259 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.767142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzrx\" (UniqueName: \"kubernetes.io/projected/a93d13a3-027d-4269-b12d-37859e1b87cc-kube-api-access-7lzrx\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.767742 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.767724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a93d13a3-027d-4269-b12d-37859e1b87cc-manager-config\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.769342 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.769322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a93d13a3-027d-4269-b12d-37859e1b87cc-metrics-cert\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.769414 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.769382    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a93d13a3-027d-4269-b12d-37859e1b87cc-cert\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.781042 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.781022    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lzrx\" (UniqueName: \"kubernetes.io/projected/a93d13a3-027d-4269-b12d-37859e1b87cc-kube-api-access-7lzrx\") pod \"lws-controller-manager-7d56c9879c-tph6q\" (UID: \"a93d13a3-027d-4269-b12d-37859e1b87cc\") " pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.826031 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.826008    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:53.963302 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:53.963276    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"]
Jul 19 14:17:53.965221 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:17:53.965192    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93d13a3_027d_4269_b12d_37859e1b87cc.slice/crio-7c277c6ec625767653072fb41108f69d8e2fb5c13b01d3ebb8dad8d2aac6e082 WatchSource:0}: Error finding container 7c277c6ec625767653072fb41108f69d8e2fb5c13b01d3ebb8dad8d2aac6e082: Status 404 returned error can't find the container with id 7c277c6ec625767653072fb41108f69d8e2fb5c13b01d3ebb8dad8d2aac6e082
Jul 19 14:17:54.763753 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:54.763716    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q" event={"ID":"a93d13a3-027d-4269-b12d-37859e1b87cc","Type":"ContainerStarted","Data":"7c277c6ec625767653072fb41108f69d8e2fb5c13b01d3ebb8dad8d2aac6e082"}
Jul 19 14:17:54.814728 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:54.814696    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:55.227798 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:55.227708    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:55.334479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:55.334448    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:55.847769 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:55.847736    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.145099 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.145052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.253149 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.253117    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.262887 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.262840    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"]
Jul 19 14:17:56.272776 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.272750    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.275125 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.275101    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 19 14:17:56.275429 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.275393    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 19 14:17:56.275429 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.275408    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q8vwm\""
Jul 19 14:17:56.275639 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.275433    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 19 14:17:56.275639 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.275465    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 19 14:17:56.281749 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.281270    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"]
Jul 19 14:17:56.377960 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.377926    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.387706 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.387662    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-tmp\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.387706 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.387704    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c24vt\" (UniqueName: \"kubernetes.io/projected/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-kube-api-access-c24vt\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.387898 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.387755    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.387898 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.387823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.387898 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.387851    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-manifests\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.478498 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.478430    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.488604 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.488581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-tmp\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.488758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.488611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c24vt\" (UniqueName: \"kubernetes.io/projected/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-kube-api-access-c24vt\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.488758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.488655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.488758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.488718    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.488758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.488746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-manifests\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.489275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.489252    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-manifests\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.490921 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.490887    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-tmp\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.491484 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.491464    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.491646 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.491623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.497280 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.497261    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c24vt\" (UniqueName: \"kubernetes.io/projected/8f85a9ec-654f-4c7b-ba7f-95ae715f41bd-kube-api-access-c24vt\") pod \"opendatahub-operator-controller-manager-bd7c6769d-62tfq\" (UID: \"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd\") " pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.578582 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.578541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.586567 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.586540    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:17:56.681973 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.681942    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.779857 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.779779    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.886632 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.886605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:17:56.907908 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:56.907811    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"]
Jul 19 14:17:56.909770 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:17:56.909740    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f85a9ec_654f_4c7b_ba7f_95ae715f41bd.slice/crio-28fc60b6d853af9da90d3c380433dc9d7b2dbf201fbe4508e2e9f6147b0aae41 WatchSource:0}: Error finding container 28fc60b6d853af9da90d3c380433dc9d7b2dbf201fbe4508e2e9f6147b0aae41: Status 404 returned error can't find the container with id 28fc60b6d853af9da90d3c380433dc9d7b2dbf201fbe4508e2e9f6147b0aae41
Jul 19 14:17:57.786700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:57.786513    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq" event={"ID":"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd","Type":"ContainerStarted","Data":"28fc60b6d853af9da90d3c380433dc9d7b2dbf201fbe4508e2e9f6147b0aae41"}
Jul 19 14:17:57.790366 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:57.790334    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q" event={"ID":"a93d13a3-027d-4269-b12d-37859e1b87cc","Type":"ContainerStarted","Data":"90b48201e207b4afba8c741e5ed02d09d4c9c62b711f04d3fd0033d5b6dbe24b"}
Jul 19 14:17:57.790576 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:57.790405    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:17:57.809840 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:57.809170    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q" podStartSLOduration=1.929071404 podStartE2EDuration="4.809151343s" totalImagesPullingTime="2.880079939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:17:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:17:57.807566814 +0000 UTC m=+296.110397830" watchObservedRunningTime="2026-07-19 14:17:57.809151343 +0000 UTC m=+296.111982361"
Jul 19 14:17:59.800188 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:17:59.800152    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq" event={"ID":"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd","Type":"ContainerStarted","Data":"499b45240716420b97ba70b0cf6a79939e2c088dc14e2817e33f973982dda42c"}
Jul 19 14:18:00.805773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:00.805736    2574 generic.go:356] "Generic (PLEG): container finished" podID="8f85a9ec-654f-4c7b-ba7f-95ae715f41bd" containerID="499b45240716420b97ba70b0cf6a79939e2c088dc14e2817e33f973982dda42c" exitCode=0
Jul 19 14:18:00.805773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:00.805774    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq" event={"ID":"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd","Type":"ContainerDied","Data":"499b45240716420b97ba70b0cf6a79939e2c088dc14e2817e33f973982dda42c"}
Jul 19 14:18:01.811625 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:01.811585    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq" event={"ID":"8f85a9ec-654f-4c7b-ba7f-95ae715f41bd","Type":"ContainerStarted","Data":"cd5c62bb05a3a9d867a0ea80591230a04e7dc8969f9d0eaa1c9cb8d63858d87b"}
Jul 19 14:18:01.811979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:01.811720    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:18:01.834456 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:01.834407    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq" podStartSLOduration=1.728618083 podStartE2EDuration="5.834393092s" totalImagesPullingTime="3.080200476s" totalInitContainerRuntime="1.025574533s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:17:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:18:01.832142389 +0000 UTC m=+300.134973397" watchObservedRunningTime="2026-07-19 14:18:01.834393092 +0000 UTC m=+300.137224106"
Jul 19 14:18:02.029629 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:02.029598    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:18:02.032077 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:02.032058    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:18:02.037039 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:02.037020    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 19 14:18:08.247539 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:08.247476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:08.347730 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:08.347692    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:08.797232 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:08.797201    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7d56c9879c-tph6q"
Jul 19 14:18:12.817688 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:12.817659    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bd7c6769d-62tfq"
Jul 19 14:18:14.039773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:14.039742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:14.097509 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:14.097480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:15.917711 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:15.917670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:16.027425 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.027394    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:16.159496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.159451    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:16.238275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.238155    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"]
Jul 19 14:18:16.338047 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.338010    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"]
Jul 19 14:18:16.338196 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.338136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.340504 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.340481    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 19 14:18:16.340738 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.340721    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tbtlp\""
Jul 19 14:18:16.341461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.341441    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 19 14:18:16.341576 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.341506    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 19 14:18:16.341644 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.341611    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 19 14:18:16.365484 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.365458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.365640 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.365503    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqlz4\" (UniqueName: \"kubernetes.io/projected/5379fabd-911c-4adc-ab08-27d4f6367d33-kube-api-access-zqlz4\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.365640 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.365595    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5379fabd-911c-4adc-ab08-27d4f6367d33-tmp\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.466332 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.466290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5379fabd-911c-4adc-ab08-27d4f6367d33-tmp\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.466496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.466355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.466496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.466398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqlz4\" (UniqueName: \"kubernetes.io/projected/5379fabd-911c-4adc-ab08-27d4f6367d33-kube-api-access-zqlz4\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.466624 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:18:16.466545    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 19 14:18:16.466624 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:18:16.466614    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs podName:5379fabd-911c-4adc-ab08-27d4f6367d33 nodeName:}" failed. No retries permitted until 2026-07-19 14:18:16.966592441 +0000 UTC m=+315.269423439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs") pod "kube-auth-proxy-5bf8c8f977-grvz4" (UID: "5379fabd-911c-4adc-ab08-27d4f6367d33") : secret "kube-auth-proxy-tls" not found
Jul 19 14:18:16.468704 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.468679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5379fabd-911c-4adc-ab08-27d4f6367d33-tmp\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.479764 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.479739    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqlz4\" (UniqueName: \"kubernetes.io/projected/5379fabd-911c-4adc-ab08-27d4f6367d33-kube-api-access-zqlz4\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.576117 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.576040    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:16.657700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.657659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:16.764165 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.764132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:16.969212 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:16.969181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:16.969648 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:18:16.969359    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 19 14:18:16.969648 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:18:16.969432    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs podName:5379fabd-911c-4adc-ab08-27d4f6367d33 nodeName:}" failed. No retries permitted until 2026-07-19 14:18:17.969412678 +0000 UTC m=+316.272243676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs") pod "kube-auth-proxy-5bf8c8f977-grvz4" (UID: "5379fabd-911c-4adc-ab08-27d4f6367d33") : secret "kube-auth-proxy-tls" not found
Jul 19 14:18:17.129135 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:17.129095    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:17.407084 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:17.407046    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:17.668208 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:17.668129    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:17.735460 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:17.735426    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:17.978235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:17.978135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:17.980751 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:17.980721    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5379fabd-911c-4adc-ab08-27d4f6367d33-tls-certs\") pod \"kube-auth-proxy-5bf8c8f977-grvz4\" (UID: \"5379fabd-911c-4adc-ab08-27d4f6367d33\") " pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:18.122900 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:18.122861    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:18.147907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:18.147874    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"
Jul 19 14:18:18.281551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:18.281375    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4"]
Jul 19 14:18:18.284166 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:18:18.284135    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5379fabd_911c_4adc_ab08_27d4f6367d33.slice/crio-fa4b4d39d4f71f41b0e9bc2333369e90c346206433b9ed234a8b2df9f755a182 WatchSource:0}: Error finding container fa4b4d39d4f71f41b0e9bc2333369e90c346206433b9ed234a8b2df9f755a182: Status 404 returned error can't find the container with id fa4b4d39d4f71f41b0e9bc2333369e90c346206433b9ed234a8b2df9f755a182
Jul 19 14:18:18.286269 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:18.286250    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:18:18.685181 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:18.685150    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:18.886392 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:18.886353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4" event={"ID":"5379fabd-911c-4adc-ab08-27d4f6367d33","Type":"ContainerStarted","Data":"fa4b4d39d4f71f41b0e9bc2333369e90c346206433b9ed234a8b2df9f755a182"}
Jul 19 14:18:21.903311 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:21.903274    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4" event={"ID":"5379fabd-911c-4adc-ab08-27d4f6367d33","Type":"ContainerStarted","Data":"20b6f38c2aebe5aa01a98b618b26415a0391bc2b29f5436ab18594df3c4549c3"}
Jul 19 14:18:21.919962 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:21.919916    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bf8c8f977-grvz4" podStartSLOduration=3.3588696479999998 podStartE2EDuration="5.919901056s" totalImagesPullingTime="2.561031408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:18:21.917812968 +0000 UTC m=+320.220643984" watchObservedRunningTime="2026-07-19 14:18:21.919901056 +0000 UTC m=+320.222732087"
Jul 19 14:18:23.238795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:23.238747    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:23.339041 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:23.339013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:23.929020 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:23.928988    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:24.204036 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:24.203956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:24.299607 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:24.299566    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:26.649667 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.649625    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:26.714754 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.714720    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"]
Jul 19 14:18:26.717857 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.717840    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:26.721061 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.721035    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-64hlk\""
Jul 19 14:18:26.721195 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.721102    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 19 14:18:26.730016 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.729990    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"]
Jul 19 14:18:26.748865 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.748814    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:26.755675 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.755626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzzrx\" (UniqueName: \"kubernetes.io/projected/78f989a3-2735-4546-b6f2-1c7f82c08499-kube-api-access-lzzrx\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:26.755849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.755689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78f989a3-2735-4546-b6f2-1c7f82c08499-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:26.856253 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.856213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzzrx\" (UniqueName: \"kubernetes.io/projected/78f989a3-2735-4546-b6f2-1c7f82c08499-kube-api-access-lzzrx\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:26.856427 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.856259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78f989a3-2735-4546-b6f2-1c7f82c08499-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:26.856427 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:18:26.856389    2574 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 19 14:18:26.856570 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:18:26.856449    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f989a3-2735-4546-b6f2-1c7f82c08499-cert podName:78f989a3-2735-4546-b6f2-1c7f82c08499 nodeName:}" failed. No retries permitted until 2026-07-19 14:18:27.356428616 +0000 UTC m=+325.659259610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/78f989a3-2735-4546-b6f2-1c7f82c08499-cert") pod "llmisvc-controller-manager-777c6bd7d5-zhkx9" (UID: "78f989a3-2735-4546-b6f2-1c7f82c08499") : secret "llmisvc-webhook-server-cert" not found
Jul 19 14:18:26.867589 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.867546    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzzrx\" (UniqueName: \"kubernetes.io/projected/78f989a3-2735-4546-b6f2-1c7f82c08499-kube-api-access-lzzrx\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:26.959986 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:26.959888    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:27.168217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:27.168177    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:27.360970 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:27.360941    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78f989a3-2735-4546-b6f2-1c7f82c08499-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:27.363281 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:27.363257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78f989a3-2735-4546-b6f2-1c7f82c08499-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zhkx9\" (UID: \"78f989a3-2735-4546-b6f2-1c7f82c08499\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:27.629629 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:27.629515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:27.760873 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:27.760836    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"]
Jul 19 14:18:27.765087 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:18:27.765058    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod78f989a3_2735_4546_b6f2_1c7f82c08499.slice/crio-85c6761186793393d84e21d270faffb6f5b6e0677c67bceac6d247ae4f5f1bf5 WatchSource:0}: Error finding container 85c6761186793393d84e21d270faffb6f5b6e0677c67bceac6d247ae4f5f1bf5: Status 404 returned error can't find the container with id 85c6761186793393d84e21d270faffb6f5b6e0677c67bceac6d247ae4f5f1bf5
Jul 19 14:18:27.936720 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:27.936602    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9" event={"ID":"78f989a3-2735-4546-b6f2-1c7f82c08499","Type":"ContainerStarted","Data":"85c6761186793393d84e21d270faffb6f5b6e0677c67bceac6d247ae4f5f1bf5"}
Jul 19 14:18:28.076210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:28.076177    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:28.168943 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:28.168912    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:30.957346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:30.957317    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9" event={"ID":"78f989a3-2735-4546-b6f2-1c7f82c08499","Type":"ContainerStarted","Data":"18dc3521a531f9aef7c17e5bc950f7a0558784873e08c07cf062aaea70b47c92"}
Jul 19 14:18:30.957735 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:30.957381    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:18:30.973784 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:30.973741    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9" podStartSLOduration=2.337565245 podStartE2EDuration="4.973725433s" totalImagesPullingTime="2.636160188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:18:30.972136446 +0000 UTC m=+329.274967466" watchObservedRunningTime="2026-07-19 14:18:30.973725433 +0000 UTC m=+329.276556497"
Jul 19 14:18:39.645346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:39.645278    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:18:39.700984 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:18:39.700953    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:01.964293 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:01.964217    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zhkx9"
Jul 19 14:19:17.371854 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:17.371807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:17.461295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:17.461254    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:17.561293 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:17.561249    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:17.863323 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:17.863284    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:17.922903 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:17.922864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:18.290560 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:18.290437    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:18.530666 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:18.530635    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:19.764538 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:19.764502    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:40.534762 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:40.534710    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:40.590250 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:40.590221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:40.678897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:40.678874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:40.730337 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:40.730284    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:41.282491 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:41.282457    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:41.336279 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:41.336247    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:51.455442 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:51.455401    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:51.961621 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:51.961577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:52.063078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:52.063028    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:19:55.467863 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:19:55.467831    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:00.114069 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:00.114020    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:00.438818 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:00.438697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:01.240994 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:01.240960    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:01.524436 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:01.524348    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:04.641198 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:04.641169    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:04.897109 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:04.897017    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:07.876371 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:07.876329    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:08.478016 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:08.477981    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:08.647640 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:08.647585    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:08.747755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:08.747678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:08.850969 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:08.850938    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:12.445277 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:12.445241    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:12.504057 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:12.504027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:18.257147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.257109    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"]
Jul 19 14:20:18.260390 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.260372    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.262900 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.262878    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 19 14:20:18.263076 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.263058    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 19 14:20:18.263377 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.263362    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z4d2t\""
Jul 19 14:20:18.278060 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.278036    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"]
Jul 19 14:20:18.329264 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.329229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b2d0bcb-6901-4267-b6ca-3a061534690d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.329416 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.329295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svzj\" (UniqueName: \"kubernetes.io/projected/1b2d0bcb-6901-4267-b6ca-3a061534690d-kube-api-access-7svzj\") pod \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.429777 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.429745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svzj\" (UniqueName: \"kubernetes.io/projected/1b2d0bcb-6901-4267-b6ca-3a061534690d-kube-api-access-7svzj\") pod \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.429961 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.429804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b2d0bcb-6901-4267-b6ca-3a061534690d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.430124 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.430103    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b2d0bcb-6901-4267-b6ca-3a061534690d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.448143 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.448122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svzj\" (UniqueName: \"kubernetes.io/projected/1b2d0bcb-6901-4267-b6ca-3a061534690d-kube-api-access-7svzj\") pod \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.569707 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.569626    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:18.714566 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.714517    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"]
Jul 19 14:20:18.716139 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:20:18.716105    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2d0bcb_6901_4267_b6ca_3a061534690d.slice/crio-476bc3cd456ed5b1b5fc1f746e0e50db0adb505e3d7ed2ea773cb3a9e0780f1d WatchSource:0}: Error finding container 476bc3cd456ed5b1b5fc1f746e0e50db0adb505e3d7ed2ea773cb3a9e0780f1d: Status 404 returned error can't find the container with id 476bc3cd456ed5b1b5fc1f746e0e50db0adb505e3d7ed2ea773cb3a9e0780f1d
Jul 19 14:20:18.734932 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.734912    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"]
Jul 19 14:20:18.751578 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.751554    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"]
Jul 19 14:20:18.785216 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.785193    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"]
Jul 19 14:20:18.789499 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.789483    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:18.821306 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.821249    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"]
Jul 19 14:20:18.934078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.934047    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926abd9d-091a-46bf-ab3e-06443562bf3c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-q5khj\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:18.934223 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:18.934089    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75vs\" (UniqueName: \"kubernetes.io/projected/926abd9d-091a-46bf-ab3e-06443562bf3c-kube-api-access-j75vs\") pod \"kuadrant-operator-controller-manager-55467d9f7-q5khj\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:19.034588 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.034559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926abd9d-091a-46bf-ab3e-06443562bf3c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-q5khj\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:19.034727 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.034597    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75vs\" (UniqueName: \"kubernetes.io/projected/926abd9d-091a-46bf-ab3e-06443562bf3c-kube-api-access-j75vs\") pod \"kuadrant-operator-controller-manager-55467d9f7-q5khj\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:19.034918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.034899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926abd9d-091a-46bf-ab3e-06443562bf3c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-q5khj\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:19.047107 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.047084    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75vs\" (UniqueName: \"kubernetes.io/projected/926abd9d-091a-46bf-ab3e-06443562bf3c-kube-api-access-j75vs\") pod \"kuadrant-operator-controller-manager-55467d9f7-q5khj\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:19.099942 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.099895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:19.231870 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.231844    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"]
Jul 19 14:20:19.251741 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:20:19.251713    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926abd9d_091a_46bf_ab3e_06443562bf3c.slice/crio-0cee963305c595c40cdf8408f3815032ff4c6dc310b351ebfe8ea940c4f09bfc WatchSource:0}: Error finding container 0cee963305c595c40cdf8408f3815032ff4c6dc310b351ebfe8ea940c4f09bfc: Status 404 returned error can't find the container with id 0cee963305c595c40cdf8408f3815032ff4c6dc310b351ebfe8ea940c4f09bfc
Jul 19 14:20:19.404370 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:19.404340    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj" event={"ID":"926abd9d-091a-46bf-ab3e-06443562bf3c","Type":"ContainerStarted","Data":"0cee963305c595c40cdf8408f3815032ff4c6dc310b351ebfe8ea940c4f09bfc"}
Jul 19 14:20:21.470789 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:21.470754    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:21.572482 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:21.572418    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:23.424671 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.424631    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" containerName="manager" containerID="cri-o://cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525" gracePeriod=2
Jul 19 14:20:23.426137 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.426114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj" event={"ID":"926abd9d-091a-46bf-ab3e-06443562bf3c","Type":"ContainerStarted","Data":"bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92"}
Jul 19 14:20:23.426265 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.426248    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:23.426897 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:23.426872    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:23.446051 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.446009    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj" podStartSLOduration=1.657370651 podStartE2EDuration="5.445980772s" totalImagesPullingTime="3.788610121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:20:23.444879883 +0000 UTC m=+441.747710900" watchObservedRunningTime="2026-07-19 14:20:23.445980772 +0000 UTC m=+441.748811788"
Jul 19 14:20:23.446733 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:23.446699    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:23.658184 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.658158    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:23.660168 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:23.660145    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:23.776012 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.775954    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b2d0bcb-6901-4267-b6ca-3a061534690d-extensions-socket-volume\") pod \"1b2d0bcb-6901-4267-b6ca-3a061534690d\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") "
Jul 19 14:20:23.776012 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.776004    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7svzj\" (UniqueName: \"kubernetes.io/projected/1b2d0bcb-6901-4267-b6ca-3a061534690d-kube-api-access-7svzj\") pod \"1b2d0bcb-6901-4267-b6ca-3a061534690d\" (UID: \"1b2d0bcb-6901-4267-b6ca-3a061534690d\") "
Jul 19 14:20:23.776245 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.776221    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2d0bcb-6901-4267-b6ca-3a061534690d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1b2d0bcb-6901-4267-b6ca-3a061534690d" (UID: "1b2d0bcb-6901-4267-b6ca-3a061534690d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:20:23.777976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.777955    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2d0bcb-6901-4267-b6ca-3a061534690d-kube-api-access-7svzj" (OuterVolumeSpecName: "kube-api-access-7svzj") pod "1b2d0bcb-6901-4267-b6ca-3a061534690d" (UID: "1b2d0bcb-6901-4267-b6ca-3a061534690d"). InnerVolumeSpecName "kube-api-access-7svzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:20:23.877173 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.877144    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b2d0bcb-6901-4267-b6ca-3a061534690d-extensions-socket-volume\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:20:23.877173 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:23.877169    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7svzj\" (UniqueName: \"kubernetes.io/projected/1b2d0bcb-6901-4267-b6ca-3a061534690d-kube-api-access-7svzj\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:20:24.396581 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.396550    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" path="/var/lib/kubelet/pods/1b2d0bcb-6901-4267-b6ca-3a061534690d/volumes"
Jul 19 14:20:24.431901 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.431877    2574 generic.go:356] "Generic (PLEG): container finished" podID="1b2d0bcb-6901-4267-b6ca-3a061534690d" containerID="cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525" exitCode=2
Jul 19 14:20:24.432216 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.431916    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:24.432216 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.431950    2574 scope.go:117] "RemoveContainer" containerID="cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525"
Jul 19 14:20:24.434094 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:24.434067    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:24.435935 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:24.435905    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-kw6md\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-kw6md"
Jul 19 14:20:24.439800 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.439783    2574 scope.go:117] "RemoveContainer" containerID="cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525"
Jul 19 14:20:24.440048 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:24.440031    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525\": container with ID starting with cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525 not found: ID does not exist" containerID="cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525"
Jul 19 14:20:24.440103 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.440058    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525"} err="failed to get container status \"cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525\": rpc error: code = NotFound desc = could not find container \"cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525\": container with ID starting with cd12e8988cd69142460a250d8833b1d815920e13b94f1e92c8d19fcca4289525 not found: ID does not exist"
Jul 19 14:20:24.460376 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.460356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:24.562517 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:24.562489    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:27.003159 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:27.003124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:27.104826 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:27.104801    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:30.364712 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:30.364680    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:30.464975 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:30.464949    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:30.856664 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:30.856592    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:30.914003 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:30.913981    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:34.434179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:34.434151    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:39.187017 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:39.186988    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:39.297043 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:39.297012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:42.194576 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:42.194512    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:42.296201 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:42.296173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:44.847176 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:44.847145    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:44.957218 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:44.957186    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:47.122733 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.122701    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"]
Jul 19 14:20:47.123199 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.122889    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj" podUID="926abd9d-091a-46bf-ab3e-06443562bf3c" containerName="manager" containerID="cri-o://bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92" gracePeriod=10
Jul 19 14:20:47.368374 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.368349    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:47.466654 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.466562    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j75vs\" (UniqueName: \"kubernetes.io/projected/926abd9d-091a-46bf-ab3e-06443562bf3c-kube-api-access-j75vs\") pod \"926abd9d-091a-46bf-ab3e-06443562bf3c\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") "
Jul 19 14:20:47.466654 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.466658    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926abd9d-091a-46bf-ab3e-06443562bf3c-extensions-socket-volume\") pod \"926abd9d-091a-46bf-ab3e-06443562bf3c\" (UID: \"926abd9d-091a-46bf-ab3e-06443562bf3c\") "
Jul 19 14:20:47.467071 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.467050    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926abd9d-091a-46bf-ab3e-06443562bf3c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "926abd9d-091a-46bf-ab3e-06443562bf3c" (UID: "926abd9d-091a-46bf-ab3e-06443562bf3c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:20:47.468677 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.468655    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926abd9d-091a-46bf-ab3e-06443562bf3c-kube-api-access-j75vs" (OuterVolumeSpecName: "kube-api-access-j75vs") pod "926abd9d-091a-46bf-ab3e-06443562bf3c" (UID: "926abd9d-091a-46bf-ab3e-06443562bf3c"). InnerVolumeSpecName "kube-api-access-j75vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:20:47.532595 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.532560    2574 generic.go:356] "Generic (PLEG): container finished" podID="926abd9d-091a-46bf-ab3e-06443562bf3c" containerID="bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92" exitCode=0
Jul 19 14:20:47.532758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.532619    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"
Jul 19 14:20:47.532758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.532638    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj" event={"ID":"926abd9d-091a-46bf-ab3e-06443562bf3c","Type":"ContainerDied","Data":"bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92"}
Jul 19 14:20:47.532758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.532692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj" event={"ID":"926abd9d-091a-46bf-ab3e-06443562bf3c","Type":"ContainerDied","Data":"0cee963305c595c40cdf8408f3815032ff4c6dc310b351ebfe8ea940c4f09bfc"}
Jul 19 14:20:47.532758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.532715    2574 scope.go:117] "RemoveContainer" containerID="bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92"
Jul 19 14:20:47.542575 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.542556    2574 scope.go:117] "RemoveContainer" containerID="bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92"
Jul 19 14:20:47.543512 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:20:47.543491    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92\": container with ID starting with bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92 not found: ID does not exist" containerID="bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92"
Jul 19 14:20:47.543619 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.543538    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92"} err="failed to get container status \"bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92\": rpc error: code = NotFound desc = could not find container \"bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92\": container with ID starting with bd0a677b63f5633301e28a0d54397c81abe2c7a659d913721db1525c05ddfe92 not found: ID does not exist"
Jul 19 14:20:47.554915 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.554887    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"]
Jul 19 14:20:47.564227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.564203    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q5khj"]
Jul 19 14:20:47.567926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.567909    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926abd9d-091a-46bf-ab3e-06443562bf3c-extensions-socket-volume\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:20:47.567978 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.567928    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j75vs\" (UniqueName: \"kubernetes.io/projected/926abd9d-091a-46bf-ab3e-06443562bf3c-kube-api-access-j75vs\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:20:47.584740 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:47.584712    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:48.396859 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:48.396816    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="926abd9d-091a-46bf-ab3e-06443562bf3c" path="/var/lib/kubelet/pods/926abd9d-091a-46bf-ab3e-06443562bf3c/volumes"
Jul 19 14:20:48.448189 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:48.448162    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:48.546047 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:48.546012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:50.819969 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:50.819922    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:50.936773 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:50.936729    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:57.590178 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:57.590147    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:57.637630 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:57.637588    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:58.611965 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:58.611905    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:20:58.710918 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:20:58.710888    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:01.596229 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:01.596195    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:01.698057 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:01.698026    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:04.335412 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.335382    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:04.336022 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336002    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:04.336336 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336317    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" containerName="manager"
Jul 19 14:21:04.336336 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336333    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" containerName="manager"
Jul 19 14:21:04.336440 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336344    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926abd9d-091a-46bf-ab3e-06443562bf3c" containerName="manager"
Jul 19 14:21:04.336440 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336349    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="926abd9d-091a-46bf-ab3e-06443562bf3c" containerName="manager"
Jul 19 14:21:04.336440 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336399    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="926abd9d-091a-46bf-ab3e-06443562bf3c" containerName="manager"
Jul 19 14:21:04.336440 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.336406    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2d0bcb-6901-4267-b6ca-3a061534690d" containerName="manager"
Jul 19 14:21:04.340946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.340932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.343539 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.343495    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 19 14:21:04.343658 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.343500    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 19 14:21:04.343808 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.343793    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 19 14:21:04.344586 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.344572    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tx7vp\""
Jul 19 14:21:04.348817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.348794    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:04.410712 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.410679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqlcd\" (UniqueName: \"kubernetes.io/projected/cd95922a-2fff-44e7-8745-3e9631f8050c-kube-api-access-rqlcd\") pod \"limitador-limitador-64c94fd64f-rjht2\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.410880 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.410741    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cd95922a-2fff-44e7-8745-3e9631f8050c-config-file\") pod \"limitador-limitador-64c94fd64f-rjht2\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.437038 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.437009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:04.450894 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.450866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:04.508757 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.508721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:04.511917 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.511887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqlcd\" (UniqueName: \"kubernetes.io/projected/cd95922a-2fff-44e7-8745-3e9631f8050c-kube-api-access-rqlcd\") pod \"limitador-limitador-64c94fd64f-rjht2\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.511996 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.511962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cd95922a-2fff-44e7-8745-3e9631f8050c-config-file\") pod \"limitador-limitador-64c94fd64f-rjht2\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.512645 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.512627    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cd95922a-2fff-44e7-8745-3e9631f8050c-config-file\") pod \"limitador-limitador-64c94fd64f-rjht2\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.519620 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.519601    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqlcd\" (UniqueName: \"kubernetes.io/projected/cd95922a-2fff-44e7-8745-3e9631f8050c-kube-api-access-rqlcd\") pod \"limitador-limitador-64c94fd64f-rjht2\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.651758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.651726    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:04.786078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:04.786043    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:04.789192 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:21:04.789168    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd95922a_2fff_44e7_8745_3e9631f8050c.slice/crio-4e0a7fdf32ec86e9200f67401508eea150b9b9d3431541181e7aac69729f60d6 WatchSource:0}: Error finding container 4e0a7fdf32ec86e9200f67401508eea150b9b9d3431541181e7aac69729f60d6: Status 404 returned error can't find the container with id 4e0a7fdf32ec86e9200f67401508eea150b9b9d3431541181e7aac69729f60d6
Jul 19 14:21:05.178863 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.178830    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:05.281003 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.280967    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:05.292236 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.292208    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-6f4xb"]
Jul 19 14:21:05.295485 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.295464    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:05.297772 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.297753    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xj4mh\""
Jul 19 14:21:05.301429 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.301404    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6f4xb"]
Jul 19 14:21:05.420804 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.420764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq8w2\" (UniqueName: \"kubernetes.io/projected/0e07fa07-c93b-4c50-b2fb-afc060a4cd41-kube-api-access-nq8w2\") pod \"authorino-75d556bd77-6f4xb\" (UID: \"0e07fa07-c93b-4c50-b2fb-afc060a4cd41\") " pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:05.521876 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.521794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq8w2\" (UniqueName: \"kubernetes.io/projected/0e07fa07-c93b-4c50-b2fb-afc060a4cd41-kube-api-access-nq8w2\") pod \"authorino-75d556bd77-6f4xb\" (UID: \"0e07fa07-c93b-4c50-b2fb-afc060a4cd41\") " pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:05.531145 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.531101    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq8w2\" (UniqueName: \"kubernetes.io/projected/0e07fa07-c93b-4c50-b2fb-afc060a4cd41-kube-api-access-nq8w2\") pod \"authorino-75d556bd77-6f4xb\" (UID: \"0e07fa07-c93b-4c50-b2fb-afc060a4cd41\") " pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:05.613169 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.613131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2" event={"ID":"cd95922a-2fff-44e7-8745-3e9631f8050c","Type":"ContainerStarted","Data":"4e0a7fdf32ec86e9200f67401508eea150b9b9d3431541181e7aac69729f60d6"}
Jul 19 14:21:05.617690 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.617460    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:05.794236 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:05.794207    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6f4xb"]
Jul 19 14:21:05.796615 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:21:05.796587    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e07fa07_c93b_4c50_b2fb_afc060a4cd41.slice/crio-4721ea0a4ee048b6fd1487d7eb6e046af3053a7f56697599590485d48e490934 WatchSource:0}: Error finding container 4721ea0a4ee048b6fd1487d7eb6e046af3053a7f56697599590485d48e490934: Status 404 returned error can't find the container with id 4721ea0a4ee048b6fd1487d7eb6e046af3053a7f56697599590485d48e490934
Jul 19 14:21:06.618482 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:06.618453    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6f4xb" event={"ID":"0e07fa07-c93b-4c50-b2fb-afc060a4cd41","Type":"ContainerStarted","Data":"4721ea0a4ee048b6fd1487d7eb6e046af3053a7f56697599590485d48e490934"}
Jul 19 14:21:07.626994 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:07.626951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2" event={"ID":"cd95922a-2fff-44e7-8745-3e9631f8050c","Type":"ContainerStarted","Data":"2854c224b3b4bf625bbe73c548f40e937d254bac07c3ace95ae2a987ceadb05f"}
Jul 19 14:21:07.627421 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:07.627162    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:07.646062 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:07.646008    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2" podStartSLOduration=0.923013328 podStartE2EDuration="3.645986641s" totalImagesPullingTime="2.722973313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:21:07.643751161 +0000 UTC m=+485.946582188" watchObservedRunningTime="2026-07-19 14:21:07.645986641 +0000 UTC m=+485.948817658"
Jul 19 14:21:09.605734 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:09.605658    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:09.639027 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:09.638987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6f4xb" event={"ID":"0e07fa07-c93b-4c50-b2fb-afc060a4cd41","Type":"ContainerStarted","Data":"c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4"}
Jul 19 14:21:09.656515 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:09.656468    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-6f4xb" podStartSLOduration=1.177024534 podStartE2EDuration="4.656451676s" totalImagesPullingTime="3.479427142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:21:09.654707738 +0000 UTC m=+487.957538756" watchObservedRunningTime="2026-07-19 14:21:09.656451676 +0000 UTC m=+487.959282691"
Jul 19 14:21:09.670201 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:09.670167    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:13.620956 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:13.620917    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:13.675511 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:13.675488    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:18.633111 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:18.633078    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:19.289952 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:19.289919    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:19.290115 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:19.290093    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2" podUID="cd95922a-2fff-44e7-8745-3e9631f8050c" containerName="limitador" containerID="cri-o://2854c224b3b4bf625bbe73c548f40e937d254bac07c3ace95ae2a987ceadb05f" gracePeriod=30
Jul 19 14:21:19.680650 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:19.680623    2574 generic.go:356] "Generic (PLEG): container finished" podID="cd95922a-2fff-44e7-8745-3e9631f8050c" containerID="2854c224b3b4bf625bbe73c548f40e937d254bac07c3ace95ae2a987ceadb05f" exitCode=0
Jul 19 14:21:19.680968 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:19.680675    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2" event={"ID":"cd95922a-2fff-44e7-8745-3e9631f8050c","Type":"ContainerDied","Data":"2854c224b3b4bf625bbe73c548f40e937d254bac07c3ace95ae2a987ceadb05f"}
Jul 19 14:21:20.231417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.231393    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:20.360422 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.360400    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqlcd\" (UniqueName: \"kubernetes.io/projected/cd95922a-2fff-44e7-8745-3e9631f8050c-kube-api-access-rqlcd\") pod \"cd95922a-2fff-44e7-8745-3e9631f8050c\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") "
Jul 19 14:21:20.360603 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.360430    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cd95922a-2fff-44e7-8745-3e9631f8050c-config-file\") pod \"cd95922a-2fff-44e7-8745-3e9631f8050c\" (UID: \"cd95922a-2fff-44e7-8745-3e9631f8050c\") "
Jul 19 14:21:20.360816 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.360791    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd95922a-2fff-44e7-8745-3e9631f8050c-config-file" (OuterVolumeSpecName: "config-file") pod "cd95922a-2fff-44e7-8745-3e9631f8050c" (UID: "cd95922a-2fff-44e7-8745-3e9631f8050c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:21:20.362418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.362390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd95922a-2fff-44e7-8745-3e9631f8050c-kube-api-access-rqlcd" (OuterVolumeSpecName: "kube-api-access-rqlcd") pod "cd95922a-2fff-44e7-8745-3e9631f8050c" (UID: "cd95922a-2fff-44e7-8745-3e9631f8050c"). InnerVolumeSpecName "kube-api-access-rqlcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:21:20.461610 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.461574    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqlcd\" (UniqueName: \"kubernetes.io/projected/cd95922a-2fff-44e7-8745-3e9631f8050c-kube-api-access-rqlcd\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:21:20.461610 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.461601    2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cd95922a-2fff-44e7-8745-3e9631f8050c-config-file\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:21:20.685993 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.685916    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2"
Jul 19 14:21:20.685993 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.685948    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rjht2" event={"ID":"cd95922a-2fff-44e7-8745-3e9631f8050c","Type":"ContainerDied","Data":"4e0a7fdf32ec86e9200f67401508eea150b9b9d3431541181e7aac69729f60d6"}
Jul 19 14:21:20.686535 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.686009    2574 scope.go:117] "RemoveContainer" containerID="2854c224b3b4bf625bbe73c548f40e937d254bac07c3ace95ae2a987ceadb05f"
Jul 19 14:21:20.701570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.701499    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:20.706710 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:20.706690    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rjht2"]
Jul 19 14:21:22.396362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:22.396331    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd95922a-2fff-44e7-8745-3e9631f8050c" path="/var/lib/kubelet/pods/cd95922a-2fff-44e7-8745-3e9631f8050c/volumes"
Jul 19 14:21:23.955128 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:23.955088    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:24.058982 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.058952    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:24.398710 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.398677    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"]
Jul 19 14:21:24.399146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.399133    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd95922a-2fff-44e7-8745-3e9631f8050c" containerName="limitador"
Jul 19 14:21:24.399187 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.399150    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd95922a-2fff-44e7-8745-3e9631f8050c" containerName="limitador"
Jul 19 14:21:24.399242 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.399232    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd95922a-2fff-44e7-8745-3e9631f8050c" containerName="limitador"
Jul 19 14:21:24.403896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.403872    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"]
Jul 19 14:21:24.404010 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.403957    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.406589 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.406566    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 14:21:24.406768 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.406744    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 19 14:21:24.407740 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.407708    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 19 14:21:24.407855 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.407723    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-k6tcj\""
Jul 19 14:21:24.422612 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.422593    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:24.596315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.596282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e16f5aa2-4435-4a66-bb88-72b4d5058da7-data\") pod \"postgres-69fb4594cf-mxcn8\" (UID: \"e16f5aa2-4435-4a66-bb88-72b4d5058da7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.596476 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.596321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpnjb\" (UniqueName: \"kubernetes.io/projected/e16f5aa2-4435-4a66-bb88-72b4d5058da7-kube-api-access-wpnjb\") pod \"postgres-69fb4594cf-mxcn8\" (UID: \"e16f5aa2-4435-4a66-bb88-72b4d5058da7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.696903 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.696825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e16f5aa2-4435-4a66-bb88-72b4d5058da7-data\") pod \"postgres-69fb4594cf-mxcn8\" (UID: \"e16f5aa2-4435-4a66-bb88-72b4d5058da7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.696903 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.696863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpnjb\" (UniqueName: \"kubernetes.io/projected/e16f5aa2-4435-4a66-bb88-72b4d5058da7-kube-api-access-wpnjb\") pod \"postgres-69fb4594cf-mxcn8\" (UID: \"e16f5aa2-4435-4a66-bb88-72b4d5058da7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.697147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.697128    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e16f5aa2-4435-4a66-bb88-72b4d5058da7-data\") pod \"postgres-69fb4594cf-mxcn8\" (UID: \"e16f5aa2-4435-4a66-bb88-72b4d5058da7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.711144 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.711114    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpnjb\" (UniqueName: \"kubernetes.io/projected/e16f5aa2-4435-4a66-bb88-72b4d5058da7-kube-api-access-wpnjb\") pod \"postgres-69fb4594cf-mxcn8\" (UID: \"e16f5aa2-4435-4a66-bb88-72b4d5058da7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.713947 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.713927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:24.838233 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:24.838211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"]
Jul 19 14:21:24.840316 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:21:24.840291    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16f5aa2_4435_4a66_bb88_72b4d5058da7.slice/crio-e34762e65e9d784f195ba64f4d805496ef26362339ba99134a8c3ba5942e2e5d WatchSource:0}: Error finding container e34762e65e9d784f195ba64f4d805496ef26362339ba99134a8c3ba5942e2e5d: Status 404 returned error can't find the container with id e34762e65e9d784f195ba64f4d805496ef26362339ba99134a8c3ba5942e2e5d
Jul 19 14:21:25.707956 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:25.707923    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8" event={"ID":"e16f5aa2-4435-4a66-bb88-72b4d5058da7","Type":"ContainerStarted","Data":"e34762e65e9d784f195ba64f4d805496ef26362339ba99134a8c3ba5942e2e5d"}
Jul 19 14:21:26.768751 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:26.768480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:30.734756 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:30.734686    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8" event={"ID":"e16f5aa2-4435-4a66-bb88-72b4d5058da7","Type":"ContainerStarted","Data":"f989e70aecb74285be2032a513d08c77ada0207837395a4130996c02830d1c7d"}
Jul 19 14:21:30.735094 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:30.734805    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:30.749664 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:30.749618    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8" podStartSLOduration=1.131373219 podStartE2EDuration="6.749607153s" totalImagesPullingTime="5.618233934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:21:30.748896302 +0000 UTC m=+509.051727355" watchObservedRunningTime="2026-07-19 14:21:30.749607153 +0000 UTC m=+509.052438169"
Jul 19 14:21:36.767536 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:36.767504    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxcn8"
Jul 19 14:21:37.078346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.078268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:37.302342 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.302312    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phzm6"]
Jul 19 14:21:37.313272 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.313229    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phzm6"]
Jul 19 14:21:37.313434 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.313395    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:37.398612 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.398579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grdqf\" (UniqueName: \"kubernetes.io/projected/0fc7a7a5-cbff-43c4-9c6f-691653596ed8-kube-api-access-grdqf\") pod \"authorino-5d4b67c446-phzm6\" (UID: \"0fc7a7a5-cbff-43c4-9c6f-691653596ed8\") " pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:37.496295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.496268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:37.499409 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.499368    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grdqf\" (UniqueName: \"kubernetes.io/projected/0fc7a7a5-cbff-43c4-9c6f-691653596ed8-kube-api-access-grdqf\") pod \"authorino-5d4b67c446-phzm6\" (UID: \"0fc7a7a5-cbff-43c4-9c6f-691653596ed8\") " pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:37.507045 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.507018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grdqf\" (UniqueName: \"kubernetes.io/projected/0fc7a7a5-cbff-43c4-9c6f-691653596ed8-kube-api-access-grdqf\") pod \"authorino-5d4b67c446-phzm6\" (UID: \"0fc7a7a5-cbff-43c4-9c6f-691653596ed8\") " pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:37.555534 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.555494    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phzm6"]
Jul 19 14:21:37.555786 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.555770    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:37.684497 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.684370    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phzm6"]
Jul 19 14:21:37.687068 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:21:37.687040    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fc7a7a5_cbff_43c4_9c6f_691653596ed8.slice/crio-b64480dac497420c3a5f6b92205cc9fe3ff108a3e89b267f5fd72b136fb3b330 WatchSource:0}: Error finding container b64480dac497420c3a5f6b92205cc9fe3ff108a3e89b267f5fd72b136fb3b330: Status 404 returned error can't find the container with id b64480dac497420c3a5f6b92205cc9fe3ff108a3e89b267f5fd72b136fb3b330
Jul 19 14:21:37.766836 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:37.766803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-phzm6" event={"ID":"0fc7a7a5-cbff-43c4-9c6f-691653596ed8","Type":"ContainerStarted","Data":"b64480dac497420c3a5f6b92205cc9fe3ff108a3e89b267f5fd72b136fb3b330"}
Jul 19 14:21:38.772270 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:38.772236    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-phzm6" event={"ID":"0fc7a7a5-cbff-43c4-9c6f-691653596ed8","Type":"ContainerStarted","Data":"7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988"}
Jul 19 14:21:38.772270 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:38.772252    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-phzm6" podUID="0fc7a7a5-cbff-43c4-9c6f-691653596ed8" containerName="authorino" containerID="cri-o://7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988" gracePeriod=30
Jul 19 14:21:38.786418 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:38.786377    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-phzm6" podStartSLOduration=1.4944410399999999 podStartE2EDuration="1.786363461s" totalImagesPullingTime="291.922421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:21:38.78572395 +0000 UTC m=+517.088554966" watchObservedRunningTime="2026-07-19 14:21:38.786363461 +0000 UTC m=+517.089194476"
Jul 19 14:21:39.015464 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.015437    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:39.111361 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.111340    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grdqf\" (UniqueName: \"kubernetes.io/projected/0fc7a7a5-cbff-43c4-9c6f-691653596ed8-kube-api-access-grdqf\") pod \"0fc7a7a5-cbff-43c4-9c6f-691653596ed8\" (UID: \"0fc7a7a5-cbff-43c4-9c6f-691653596ed8\") "
Jul 19 14:21:39.113250 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.113227    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc7a7a5-cbff-43c4-9c6f-691653596ed8-kube-api-access-grdqf" (OuterVolumeSpecName: "kube-api-access-grdqf") pod "0fc7a7a5-cbff-43c4-9c6f-691653596ed8" (UID: "0fc7a7a5-cbff-43c4-9c6f-691653596ed8"). InnerVolumeSpecName "kube-api-access-grdqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:21:39.212430 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.212402    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grdqf\" (UniqueName: \"kubernetes.io/projected/0fc7a7a5-cbff-43c4-9c6f-691653596ed8-kube-api-access-grdqf\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:21:39.778281 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.778241    2574 generic.go:356] "Generic (PLEG): container finished" podID="0fc7a7a5-cbff-43c4-9c6f-691653596ed8" containerID="7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988" exitCode=0
Jul 19 14:21:39.778755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.778341    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phzm6"
Jul 19 14:21:39.778755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.778417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-phzm6" event={"ID":"0fc7a7a5-cbff-43c4-9c6f-691653596ed8","Type":"ContainerDied","Data":"7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988"}
Jul 19 14:21:39.778755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.778558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-phzm6" event={"ID":"0fc7a7a5-cbff-43c4-9c6f-691653596ed8","Type":"ContainerDied","Data":"b64480dac497420c3a5f6b92205cc9fe3ff108a3e89b267f5fd72b136fb3b330"}
Jul 19 14:21:39.778755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.778591    2574 scope.go:117] "RemoveContainer" containerID="7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988"
Jul 19 14:21:39.781554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.781515    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6f4xb"]
Jul 19 14:21:39.781735 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.781714    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-6f4xb" podUID="0e07fa07-c93b-4c50-b2fb-afc060a4cd41" containerName="authorino" containerID="cri-o://c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4" gracePeriod=30
Jul 19 14:21:39.789297 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.789058    2574 scope.go:117] "RemoveContainer" containerID="7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988"
Jul 19 14:21:39.789439 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:21:39.789410    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988\": container with ID starting with 7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988 not found: ID does not exist" containerID="7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988"
Jul 19 14:21:39.789514 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.789444    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988"} err="failed to get container status \"7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988\": rpc error: code = NotFound desc = could not find container \"7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988\": container with ID starting with 7124d1df73fad3069b06b82c84f900df8fd73fc3888e4e2f98d0be96a0842988 not found: ID does not exist"
Jul 19 14:21:39.807767 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.807738    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phzm6"]
Jul 19 14:21:39.810278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:39.810259    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phzm6"]
Jul 19 14:21:40.028502 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.028444    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:40.118290 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.118265    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq8w2\" (UniqueName: \"kubernetes.io/projected/0e07fa07-c93b-4c50-b2fb-afc060a4cd41-kube-api-access-nq8w2\") pod \"0e07fa07-c93b-4c50-b2fb-afc060a4cd41\" (UID: \"0e07fa07-c93b-4c50-b2fb-afc060a4cd41\") "
Jul 19 14:21:40.120164 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.120136    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e07fa07-c93b-4c50-b2fb-afc060a4cd41-kube-api-access-nq8w2" (OuterVolumeSpecName: "kube-api-access-nq8w2") pod "0e07fa07-c93b-4c50-b2fb-afc060a4cd41" (UID: "0e07fa07-c93b-4c50-b2fb-afc060a4cd41"). InnerVolumeSpecName "kube-api-access-nq8w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:21:40.219697 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.219668    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq8w2\" (UniqueName: \"kubernetes.io/projected/0e07fa07-c93b-4c50-b2fb-afc060a4cd41-kube-api-access-nq8w2\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:21:40.397352 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.397310    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fc7a7a5-cbff-43c4-9c6f-691653596ed8" path="/var/lib/kubelet/pods/0fc7a7a5-cbff-43c4-9c6f-691653596ed8/volumes"
Jul 19 14:21:40.784189 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.784162    2574 generic.go:356] "Generic (PLEG): container finished" podID="0e07fa07-c93b-4c50-b2fb-afc060a4cd41" containerID="c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4" exitCode=0
Jul 19 14:21:40.784592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.784200    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6f4xb"
Jul 19 14:21:40.784592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.784246    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6f4xb" event={"ID":"0e07fa07-c93b-4c50-b2fb-afc060a4cd41","Type":"ContainerDied","Data":"c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4"}
Jul 19 14:21:40.784592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.784301    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6f4xb" event={"ID":"0e07fa07-c93b-4c50-b2fb-afc060a4cd41","Type":"ContainerDied","Data":"4721ea0a4ee048b6fd1487d7eb6e046af3053a7f56697599590485d48e490934"}
Jul 19 14:21:40.784592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.784331    2574 scope.go:117] "RemoveContainer" containerID="c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4"
Jul 19 14:21:40.792253 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.792239    2574 scope.go:117] "RemoveContainer" containerID="c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4"
Jul 19 14:21:40.792463 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:21:40.792442    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4\": container with ID starting with c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4 not found: ID does not exist" containerID="c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4"
Jul 19 14:21:40.792611 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.792474    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4"} err="failed to get container status \"c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4\": rpc error: code = NotFound desc = could not find container \"c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4\": container with ID starting with c5f1463419d8e84c74fa8ce100d9709bc248c01ebfab55ff38561435135c8ab4 not found: ID does not exist"
Jul 19 14:21:40.799142 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.799119    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6f4xb"]
Jul 19 14:21:40.800454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:40.800436    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6f4xb"]
Jul 19 14:21:42.396187 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:42.396147    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e07fa07-c93b-4c50-b2fb-afc060a4cd41" path="/var/lib/kubelet/pods/0e07fa07-c93b-4c50-b2fb-afc060a4cd41/volumes"
Jul 19 14:21:44.500895 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.500852    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:44.512836 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.512806    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b88759b9c-t92s4"]
Jul 19 14:21:44.513130 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.513119    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e07fa07-c93b-4c50-b2fb-afc060a4cd41" containerName="authorino"
Jul 19 14:21:44.513186 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.513132    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e07fa07-c93b-4c50-b2fb-afc060a4cd41" containerName="authorino"
Jul 19 14:21:44.513186 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.513145    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fc7a7a5-cbff-43c4-9c6f-691653596ed8" containerName="authorino"
Jul 19 14:21:44.513186 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.513150    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fc7a7a5-cbff-43c4-9c6f-691653596ed8" containerName="authorino"
Jul 19 14:21:44.513277 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.513208    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e07fa07-c93b-4c50-b2fb-afc060a4cd41" containerName="authorino"
Jul 19 14:21:44.513277 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.513218    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fc7a7a5-cbff-43c4-9c6f-691653596ed8" containerName="authorino"
Jul 19 14:21:44.517796 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.517777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:44.520311 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.520289    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 19 14:21:44.520411 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.520352    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2jtlk\""
Jul 19 14:21:44.520462 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.520410    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 19 14:21:44.526905 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.526885    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b88759b9c-t92s4"]
Jul 19 14:21:44.551772 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.551738    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:44.551911 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.551814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc6ft\" (UniqueName: \"kubernetes.io/projected/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-kube-api-access-fc6ft\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:44.600593 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.600567    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:44.652879 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.652848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc6ft\" (UniqueName: \"kubernetes.io/projected/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-kube-api-access-fc6ft\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:44.653044 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.652914    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:44.653044 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:21:44.653025    2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 19 14:21:44.653161 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:21:44.653104    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert podName:77e6c37e-6ebe-44d6-93f4-84c09cf91a54 nodeName:}" failed. No retries permitted until 2026-07-19 14:21:45.153080153 +0000 UTC m=+523.455911151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert") pod "maas-controller-7b88759b9c-t92s4" (UID: "77e6c37e-6ebe-44d6-93f4-84c09cf91a54") : secret "maas-controller-webhook-cert" not found
Jul 19 14:21:44.655998 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.655975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:44.661395 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.661375    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc6ft\" (UniqueName: \"kubernetes.io/projected/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-kube-api-access-fc6ft\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:44.694398 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.694371    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57866856f9-bvqn8"]
Jul 19 14:21:44.696639 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.696617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:44.706832 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.706804    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57866856f9-bvqn8"]
Jul 19 14:21:44.725230 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.725205    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:44.753577 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.753495    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq2xh\" (UniqueName: \"kubernetes.io/projected/0078ab5a-2a09-4107-9e24-e69e5a429c81-kube-api-access-wq2xh\") pod \"maas-controller-57866856f9-bvqn8\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") " pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:44.753690 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.753655    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0078ab5a-2a09-4107-9e24-e69e5a429c81-webhook-cert\") pod \"maas-controller-57866856f9-bvqn8\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") " pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:44.854750 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.854720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0078ab5a-2a09-4107-9e24-e69e5a429c81-webhook-cert\") pod \"maas-controller-57866856f9-bvqn8\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") " pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:44.854939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.854770    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2xh\" (UniqueName: \"kubernetes.io/projected/0078ab5a-2a09-4107-9e24-e69e5a429c81-kube-api-access-wq2xh\") pod \"maas-controller-57866856f9-bvqn8\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") " pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:44.857203 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.857177    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0078ab5a-2a09-4107-9e24-e69e5a429c81-webhook-cert\") pod \"maas-controller-57866856f9-bvqn8\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") " pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:44.862751 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:44.862720    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq2xh\" (UniqueName: \"kubernetes.io/projected/0078ab5a-2a09-4107-9e24-e69e5a429c81-kube-api-access-wq2xh\") pod \"maas-controller-57866856f9-bvqn8\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") " pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:45.007355 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.007262    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:45.131614 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.131553    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57866856f9-bvqn8"]
Jul 19 14:21:45.133968 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:21:45.133944    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0078ab5a_2a09_4107_9e24_e69e5a429c81.slice/crio-0a23a879a50f64a8104dbf2e56e3049b178a6f28ee173de70f4c70251e7b5901 WatchSource:0}: Error finding container 0a23a879a50f64a8104dbf2e56e3049b178a6f28ee173de70f4c70251e7b5901: Status 404 returned error can't find the container with id 0a23a879a50f64a8104dbf2e56e3049b178a6f28ee173de70f4c70251e7b5901
Jul 19 14:21:45.157164 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.157144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:45.159308 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.159291    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert\") pod \"maas-controller-7b88759b9c-t92s4\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") " pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:45.430051 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.430022    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:45.552938 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.552912    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b88759b9c-t92s4"]
Jul 19 14:21:45.554665 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:21:45.554638    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e6c37e_6ebe_44d6_93f4_84c09cf91a54.slice/crio-e4cab4d7ba02d4ed929d79004b5c293aa8081be822d91d008ce6bc612bc1a904 WatchSource:0}: Error finding container e4cab4d7ba02d4ed929d79004b5c293aa8081be822d91d008ce6bc612bc1a904: Status 404 returned error can't find the container with id e4cab4d7ba02d4ed929d79004b5c293aa8081be822d91d008ce6bc612bc1a904
Jul 19 14:21:45.815079 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.814997    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b88759b9c-t92s4" event={"ID":"77e6c37e-6ebe-44d6-93f4-84c09cf91a54","Type":"ContainerStarted","Data":"e4cab4d7ba02d4ed929d79004b5c293aa8081be822d91d008ce6bc612bc1a904"}
Jul 19 14:21:45.816588 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:45.816557    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-bvqn8" event={"ID":"0078ab5a-2a09-4107-9e24-e69e5a429c81","Type":"ContainerStarted","Data":"0a23a879a50f64a8104dbf2e56e3049b178a6f28ee173de70f4c70251e7b5901"}
Jul 19 14:21:47.621511 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.621479    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:47.728953 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.728871    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:21:47.829238 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.829207    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b88759b9c-t92s4" event={"ID":"77e6c37e-6ebe-44d6-93f4-84c09cf91a54","Type":"ContainerStarted","Data":"de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7"}
Jul 19 14:21:47.829434 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.829326    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:47.830881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.830859    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-bvqn8" event={"ID":"0078ab5a-2a09-4107-9e24-e69e5a429c81","Type":"ContainerStarted","Data":"b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467"}
Jul 19 14:21:47.830978 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.830907    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:47.849064 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.849018    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b88759b9c-t92s4" podStartSLOduration=1.928810841 podStartE2EDuration="3.849004862s" totalImagesPullingTime="1.920194021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:21:47.848770318 +0000 UTC m=+526.151601359" watchObservedRunningTime="2026-07-19 14:21:47.849004862 +0000 UTC m=+526.151835876"
Jul 19 14:21:47.867100 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:47.867058    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57866856f9-bvqn8" podStartSLOduration=1.532463742 podStartE2EDuration="3.86704355s" totalImagesPullingTime="2.334579808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:21:47.865758515 +0000 UTC m=+526.168589533" watchObservedRunningTime="2026-07-19 14:21:47.86704355 +0000 UTC m=+526.169874565"
Jul 19 14:21:58.837702 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:58.837629    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:21:58.838209 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:58.837705    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:58.897873 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:58.897843    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b88759b9c-t92s4"]
Jul 19 14:21:58.898051 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:58.898033    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b88759b9c-t92s4" podUID="77e6c37e-6ebe-44d6-93f4-84c09cf91a54" containerName="manager" containerID="cri-o://de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7" gracePeriod=30
Jul 19 14:21:59.145989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.145962    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:59.292304 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.292269    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert\") pod \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") "
Jul 19 14:21:59.292495 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.292368    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc6ft\" (UniqueName: \"kubernetes.io/projected/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-kube-api-access-fc6ft\") pod \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\" (UID: \"77e6c37e-6ebe-44d6-93f4-84c09cf91a54\") "
Jul 19 14:21:59.294365 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.294338    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "77e6c37e-6ebe-44d6-93f4-84c09cf91a54" (UID: "77e6c37e-6ebe-44d6-93f4-84c09cf91a54"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:21:59.294482 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.294426    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-kube-api-access-fc6ft" (OuterVolumeSpecName: "kube-api-access-fc6ft") pod "77e6c37e-6ebe-44d6-93f4-84c09cf91a54" (UID: "77e6c37e-6ebe-44d6-93f4-84c09cf91a54"). InnerVolumeSpecName "kube-api-access-fc6ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:21:59.393907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.393816    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-webhook-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:21:59.393907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.393857    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc6ft\" (UniqueName: \"kubernetes.io/projected/77e6c37e-6ebe-44d6-93f4-84c09cf91a54-kube-api-access-fc6ft\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:21:59.890227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.890197    2574 generic.go:356] "Generic (PLEG): container finished" podID="77e6c37e-6ebe-44d6-93f4-84c09cf91a54" containerID="de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7" exitCode=0
Jul 19 14:21:59.890698 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.890244    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b88759b9c-t92s4"
Jul 19 14:21:59.890698 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.890282    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b88759b9c-t92s4" event={"ID":"77e6c37e-6ebe-44d6-93f4-84c09cf91a54","Type":"ContainerDied","Data":"de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7"}
Jul 19 14:21:59.890698 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.890347    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b88759b9c-t92s4" event={"ID":"77e6c37e-6ebe-44d6-93f4-84c09cf91a54","Type":"ContainerDied","Data":"e4cab4d7ba02d4ed929d79004b5c293aa8081be822d91d008ce6bc612bc1a904"}
Jul 19 14:21:59.890698 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.890382    2574 scope.go:117] "RemoveContainer" containerID="de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7"
Jul 19 14:21:59.899888 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.899693    2574 scope.go:117] "RemoveContainer" containerID="de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7"
Jul 19 14:21:59.900005 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:21:59.899982    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7\": container with ID starting with de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7 not found: ID does not exist" containerID="de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7"
Jul 19 14:21:59.900045 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.900019    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7"} err="failed to get container status \"de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7\": rpc error: code = NotFound desc = could not find container \"de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7\": container with ID starting with de58a3d8681499d7f51f0181d352ee485265f02054b0aa4d9e350a6ad550eec7 not found: ID does not exist"
Jul 19 14:21:59.912047 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.912027    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b88759b9c-t92s4"]
Jul 19 14:21:59.920624 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:21:59.915780    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b88759b9c-t92s4"]
Jul 19 14:22:00.396631 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:00.396593    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77e6c37e-6ebe-44d6-93f4-84c09cf91a54" path="/var/lib/kubelet/pods/77e6c37e-6ebe-44d6-93f4-84c09cf91a54/volumes"
Jul 19 14:22:17.555319 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:17.555276    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:17.652579 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:17.652541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:18.138286 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.138257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:18.286939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.286908    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-9b85996-7tvq2"]
Jul 19 14:22:18.287242 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.287230    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77e6c37e-6ebe-44d6-93f4-84c09cf91a54" containerName="manager"
Jul 19 14:22:18.287242 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.287242    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="77e6c37e-6ebe-44d6-93f4-84c09cf91a54" containerName="manager"
Jul 19 14:22:18.287348 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.287306    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="77e6c37e-6ebe-44d6-93f4-84c09cf91a54" containerName="manager"
Jul 19 14:22:18.315709 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.315681    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-9b85996-7tvq2"]
Jul 19 14:22:18.315885 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.315793    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.318386 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.318360    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 19 14:22:18.318501 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.318433    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6hlc4\""
Jul 19 14:22:18.357213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.357188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e9afaf-db98-4f0a-8a7e-7a631de58076-maas-api-tls\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.357362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.357228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wgjx\" (UniqueName: \"kubernetes.io/projected/c1e9afaf-db98-4f0a-8a7e-7a631de58076-kube-api-access-7wgjx\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.357362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.357267    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e9afaf-db98-4f0a-8a7e-7a631de58076-openshift-service-ca\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.437414 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.437326    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:18.457790 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.457756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e9afaf-db98-4f0a-8a7e-7a631de58076-maas-api-tls\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.457979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.457838    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wgjx\" (UniqueName: \"kubernetes.io/projected/c1e9afaf-db98-4f0a-8a7e-7a631de58076-kube-api-access-7wgjx\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.457979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.457925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e9afaf-db98-4f0a-8a7e-7a631de58076-openshift-service-ca\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.458643 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.458625    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e9afaf-db98-4f0a-8a7e-7a631de58076-openshift-service-ca\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.460277 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.460255    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e9afaf-db98-4f0a-8a7e-7a631de58076-maas-api-tls\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.467096 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.467076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wgjx\" (UniqueName: \"kubernetes.io/projected/c1e9afaf-db98-4f0a-8a7e-7a631de58076-kube-api-access-7wgjx\") pod \"maas-api-9b85996-7tvq2\" (UID: \"c1e9afaf-db98-4f0a-8a7e-7a631de58076\") " pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.626191 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.626157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:18.636228 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.636199    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:18.748928 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.746457    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"]
Jul 19 14:22:18.748928 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.747233    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:18.752920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.752902    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:18.756794 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.756279    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lf28p\""
Jul 19 14:22:18.756794 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.756515    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 19 14:22:18.760615 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.760576    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"]
Jul 19 14:22:18.761753 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.761735    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ndxvw"]
Jul 19 14:22:18.765708 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.765678    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.775469 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.775398    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ndxvw"]
Jul 19 14:22:18.788065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.788041    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-9b85996-7tvq2"]
Jul 19 14:22:18.791696 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:22:18.791670    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e9afaf_db98_4f0a_8a7e_7a631de58076.slice/crio-e7d3ebf2dcfe0668bf3826d84b801de675ae838880a01a49cfb667e92ff3e456 WatchSource:0}: Error finding container e7d3ebf2dcfe0668bf3826d84b801de675ae838880a01a49cfb667e92ff3e456: Status 404 returned error can't find the container with id e7d3ebf2dcfe0668bf3826d84b801de675ae838880a01a49cfb667e92ff3e456
Jul 19 14:22:18.862124 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.862093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e0cc04-0363-4041-a1e6-aafb1513b74e-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ndxvw\" (UID: \"79e0cc04-0363-4041-a1e6-aafb1513b74e\") " pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.862287 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.862168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9bc7\" (UniqueName: \"kubernetes.io/projected/841db817-1a2c-4981-99e3-31c9b3d10f98-kube-api-access-d9bc7\") pod \"payload-pre-processing-7677b774bb-jrzjx\" (UID: \"841db817-1a2c-4981-99e3-31c9b3d10f98\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:18.862287 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.862241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841db817-1a2c-4981-99e3-31c9b3d10f98-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jrzjx\" (UID: \"841db817-1a2c-4981-99e3-31c9b3d10f98\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:18.862287 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.862278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9jw6\" (UniqueName: \"kubernetes.io/projected/79e0cc04-0363-4041-a1e6-aafb1513b74e-kube-api-access-r9jw6\") pod \"payload-processing-5bf47cd988-ndxvw\" (UID: \"79e0cc04-0363-4041-a1e6-aafb1513b74e\") " pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.963720 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.963635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e0cc04-0363-4041-a1e6-aafb1513b74e-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ndxvw\" (UID: \"79e0cc04-0363-4041-a1e6-aafb1513b74e\") " pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.963897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.963734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9bc7\" (UniqueName: \"kubernetes.io/projected/841db817-1a2c-4981-99e3-31c9b3d10f98-kube-api-access-d9bc7\") pod \"payload-pre-processing-7677b774bb-jrzjx\" (UID: \"841db817-1a2c-4981-99e3-31c9b3d10f98\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:18.963897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.963763    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841db817-1a2c-4981-99e3-31c9b3d10f98-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jrzjx\" (UID: \"841db817-1a2c-4981-99e3-31c9b3d10f98\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:18.963897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.963780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jw6\" (UniqueName: \"kubernetes.io/projected/79e0cc04-0363-4041-a1e6-aafb1513b74e-kube-api-access-r9jw6\") pod \"payload-processing-5bf47cd988-ndxvw\" (UID: \"79e0cc04-0363-4041-a1e6-aafb1513b74e\") " pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.964416 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.964388    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e0cc04-0363-4041-a1e6-aafb1513b74e-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ndxvw\" (UID: \"79e0cc04-0363-4041-a1e6-aafb1513b74e\") " pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.964684 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.964666    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841db817-1a2c-4981-99e3-31c9b3d10f98-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-jrzjx\" (UID: \"841db817-1a2c-4981-99e3-31c9b3d10f98\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:18.970279 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.970236    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2" event={"ID":"c1e9afaf-db98-4f0a-8a7e-7a631de58076","Type":"ContainerStarted","Data":"e7d3ebf2dcfe0668bf3826d84b801de675ae838880a01a49cfb667e92ff3e456"}
Jul 19 14:22:18.974675 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.974650    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jw6\" (UniqueName: \"kubernetes.io/projected/79e0cc04-0363-4041-a1e6-aafb1513b74e-kube-api-access-r9jw6\") pod \"payload-processing-5bf47cd988-ndxvw\" (UID: \"79e0cc04-0363-4041-a1e6-aafb1513b74e\") " pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:18.974817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:18.974796    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9bc7\" (UniqueName: \"kubernetes.io/projected/841db817-1a2c-4981-99e3-31c9b3d10f98-kube-api-access-d9bc7\") pod \"payload-pre-processing-7677b774bb-jrzjx\" (UID: \"841db817-1a2c-4981-99e3-31c9b3d10f98\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:19.066915 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:19.066885    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:19.077182 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:19.077156    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:19.412602 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:19.412509    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"]
Jul 19 14:22:19.414908 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:22:19.414879    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841db817_1a2c_4981_99e3_31c9b3d10f98.slice/crio-a62cd9bda1f114d46c46290ed575dc2313fbc5618814f958c045ff5e8f95528d WatchSource:0}: Error finding container a62cd9bda1f114d46c46290ed575dc2313fbc5618814f958c045ff5e8f95528d: Status 404 returned error can't find the container with id a62cd9bda1f114d46c46290ed575dc2313fbc5618814f958c045ff5e8f95528d
Jul 19 14:22:19.438082 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:19.438060    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ndxvw"]
Jul 19 14:22:19.440923 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:22:19.440896    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e0cc04_0363_4041_a1e6_aafb1513b74e.slice/crio-c668637dc98b6a2432dc65621136b2869449ca99741e4140e2283cc098fa883e WatchSource:0}: Error finding container c668637dc98b6a2432dc65621136b2869449ca99741e4140e2283cc098fa883e: Status 404 returned error can't find the container with id c668637dc98b6a2432dc65621136b2869449ca99741e4140e2283cc098fa883e
Jul 19 14:22:19.976708 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:19.976661    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw" event={"ID":"79e0cc04-0363-4041-a1e6-aafb1513b74e","Type":"ContainerStarted","Data":"c668637dc98b6a2432dc65621136b2869449ca99741e4140e2283cc098fa883e"}
Jul 19 14:22:19.977998 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:19.977964    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx" event={"ID":"841db817-1a2c-4981-99e3-31c9b3d10f98","Type":"ContainerStarted","Data":"a62cd9bda1f114d46c46290ed575dc2313fbc5618814f958c045ff5e8f95528d"}
Jul 19 14:22:20.819147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:20.819092    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:20.986763 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:20.986564    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2" event={"ID":"c1e9afaf-db98-4f0a-8a7e-7a631de58076","Type":"ContainerStarted","Data":"849db743361403979530753bbb844e9d52bc19a87cf061989b836b584816345a"}
Jul 19 14:22:20.987186 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:20.986921    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:21.004193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:21.004147    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2" podStartSLOduration=1.63874575 podStartE2EDuration="3.004129981s" totalImagesPullingTime="1.365384231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:22:21.001957294 +0000 UTC m=+559.304788311" watchObservedRunningTime="2026-07-19 14:22:21.004129981 +0000 UTC m=+559.306961002"
Jul 19 14:22:22.997183 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:22.997144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw" event={"ID":"79e0cc04-0363-4041-a1e6-aafb1513b74e","Type":"ContainerStarted","Data":"7b8de57d5a79dc27fc69f2ec9b6dca7e7b470a951ce69c768876477e027636cc"}
Jul 19 14:22:22.997635 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:22.997276    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:22.998556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:22.998532    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx" event={"ID":"841db817-1a2c-4981-99e3-31c9b3d10f98","Type":"ContainerStarted","Data":"8f61f4651000aec60ad387b20025ec6c88091a929eb05d383128183ec093fe2c"}
Jul 19 14:22:22.998663 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:22.998636    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:23.012018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:23.011979    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw" podStartSLOduration=2.322205507 podStartE2EDuration="5.011964916s" totalImagesPullingTime="2.689759409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:22:23.011262579 +0000 UTC m=+561.314093596" watchObservedRunningTime="2026-07-19 14:22:23.011964916 +0000 UTC m=+561.314795930"
Jul 19 14:22:23.025275 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:23.025243    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx" podStartSLOduration=2.315447511 podStartE2EDuration="5.025232649s" totalImagesPullingTime="2.709785138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:22:23.024495981 +0000 UTC m=+561.327327000" watchObservedRunningTime="2026-07-19 14:22:23.025232649 +0000 UTC m=+561.328063665"
Jul 19 14:22:26.997260 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:26.997233    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-9b85996-7tvq2"
Jul 19 14:22:27.996265 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:27.996235    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:28.100723 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:28.100678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:34.004695 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:34.004664    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-ndxvw"
Jul 19 14:22:34.005280 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:34.004807    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-jrzjx"
Jul 19 14:22:38.913987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.913955    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"]
Jul 19 14:22:38.918042 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.918022    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:38.920753 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.920732    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 19 14:22:38.921791 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.921751    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vrhz7\""
Jul 19 14:22:38.921911 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.921808    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 19 14:22:38.921911 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.921759    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 19 14:22:38.926410 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.926114    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"]
Jul 19 14:22:38.946907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:38.946882    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:39.035788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.035755    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.035788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.035792    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f09a39-2f4c-44db-b718-bce84a492038-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.035989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.035835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxtn\" (UniqueName: \"kubernetes.io/projected/10f09a39-2f4c-44db-b718-bce84a492038-kube-api-access-2dxtn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.035989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.035875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.035989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.035916    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.035989 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.035950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxtn\" (UniqueName: \"kubernetes.io/projected/10f09a39-2f4c-44db-b718-bce84a492038-kube-api-access-2dxtn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137348    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137412    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f09a39-2f4c-44db-b718-bce84a492038-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137788 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137697    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137837 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.137837 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.137787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.139636 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.139617    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10f09a39-2f4c-44db-b718-bce84a492038-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.139812 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.139795    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f09a39-2f4c-44db-b718-bce84a492038-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.144752 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.144724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxtn\" (UniqueName: \"kubernetes.io/projected/10f09a39-2f4c-44db-b718-bce84a492038-kube-api-access-2dxtn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z\" (UID: \"10f09a39-2f4c-44db-b718-bce84a492038\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.229827 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.229763    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:39.360844 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:39.360822    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"]
Jul 19 14:22:39.363649 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:22:39.363623    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f09a39_2f4c_44db_b718_bce84a492038.slice/crio-806c227a28f202a64f775645cda863066bff231db0e60e98b1a337dc615cbdd1 WatchSource:0}: Error finding container 806c227a28f202a64f775645cda863066bff231db0e60e98b1a337dc615cbdd1: Status 404 returned error can't find the container with id 806c227a28f202a64f775645cda863066bff231db0e60e98b1a337dc615cbdd1
Jul 19 14:22:40.078239 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:40.078201    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z" event={"ID":"10f09a39-2f4c-44db-b718-bce84a492038","Type":"ContainerStarted","Data":"806c227a28f202a64f775645cda863066bff231db0e60e98b1a337dc615cbdd1"}
Jul 19 14:22:41.084865 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:41.084832    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z" event={"ID":"10f09a39-2f4c-44db-b718-bce84a492038","Type":"ContainerStarted","Data":"084e0905f117a8c3bbdfc6f99dc292898ccd6e7aeecaa6f84a3b024938112d61"}
Jul 19 14:22:41.101896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:41.101848    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z" podStartSLOduration=1.916573567 podStartE2EDuration="3.10183314s" totalImagesPullingTime="1.185259573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:22:41.100254593 +0000 UTC m=+579.403085608" watchObservedRunningTime="2026-07-19 14:22:41.10183314 +0000 UTC m=+579.404664185"
Jul 19 14:22:47.326363 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:47.326331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:22:49.230706 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:49.230669    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:49.231085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:49.230775    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:49.242583 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:49.242559    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:50.140636 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:50.140608    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z"
Jul 19 14:22:57.231327 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:22:57.231296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:23:00.800727 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.800699    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"]
Jul 19 14:23:00.804346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.804326    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:00.806867 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.806850    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 19 14:23:00.814363 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.814342    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"]
Jul 19 14:23:00.839009 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.838983    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:23:00.922737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.922698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:00.922910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.922761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2d26eae-d1df-4e63-850b-6614ef860290-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:00.922910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.922806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:00.922910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.922823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5vhn\" (UniqueName: \"kubernetes.io/projected/b2d26eae-d1df-4e63-850b-6614ef860290-kube-api-access-j5vhn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:00.922910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.922853    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:00.922910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:00.922870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2d26eae-d1df-4e63-850b-6614ef860290-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5vhn\" (UniqueName: \"kubernetes.io/projected/b2d26eae-d1df-4e63-850b-6614ef860290-kube-api-access-j5vhn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024647 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024761 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024763    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.024944 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.024919    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.026558 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.026514    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2d26eae-d1df-4e63-850b-6614ef860290-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.026797 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.026778    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2d26eae-d1df-4e63-850b-6614ef860290-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.031766 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.031746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5vhn\" (UniqueName: \"kubernetes.io/projected/b2d26eae-d1df-4e63-850b-6614ef860290-kube-api-access-j5vhn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx\" (UID: \"b2d26eae-d1df-4e63-850b-6614ef860290\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.115216 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.115180    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:01.242167 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:01.242141    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"]
Jul 19 14:23:01.244300 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:23:01.244261    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d26eae_d1df_4e63_850b_6614ef860290.slice/crio-7fecde17a43c66afa513442a3243867622a7b3437445e4366bf74c8404edd5b9 WatchSource:0}: Error finding container 7fecde17a43c66afa513442a3243867622a7b3437445e4366bf74c8404edd5b9: Status 404 returned error can't find the container with id 7fecde17a43c66afa513442a3243867622a7b3437445e4366bf74c8404edd5b9
Jul 19 14:23:02.060481 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:02.060453    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:23:02.061172 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:02.061155    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:23:02.185996 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:02.185965    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx" event={"ID":"b2d26eae-d1df-4e63-850b-6614ef860290","Type":"ContainerStarted","Data":"83773d3590eb981ad8874dde1e7a96e49f2d5cccaf12238c68be37e273365b43"}
Jul 19 14:23:02.186150 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:02.186037    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx" event={"ID":"b2d26eae-d1df-4e63-850b-6614ef860290","Type":"ContainerStarted","Data":"7fecde17a43c66afa513442a3243867622a7b3437445e4366bf74c8404edd5b9"}
Jul 19 14:23:02.203063 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:02.203012    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx" podStartSLOduration=2.029091526 podStartE2EDuration="2.2029979s" totalImagesPullingTime="173.906374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:23:02.202192034 +0000 UTC m=+600.505023052" watchObservedRunningTime="2026-07-19 14:23:02.2029979 +0000 UTC m=+600.505828916"
Jul 19 14:23:04.929807 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:04.929770    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:23:11.115490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:11.115452    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:11.121017 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:11.115518    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:11.127849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:11.127827    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:11.239327 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:11.239293    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx"
Jul 19 14:23:20.537622 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:23:20.537590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:24:08.873823 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:24:08.873793    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:24:08.927348 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:24:08.927325    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:24:17.399427 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:24:17.399387    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:24:17.501786 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:24:17.501753    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:24:18.545271 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:24:18.545223    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:24:18.645042 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:24:18.645008    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:25:03.667294 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.667213    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57866856f9-bvqn8"]
Jul 19 14:25:03.667827 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.667441    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57866856f9-bvqn8" podUID="0078ab5a-2a09-4107-9e24-e69e5a429c81" containerName="manager" containerID="cri-o://b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467" gracePeriod=30
Jul 19 14:25:03.906829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.906808    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:25:03.955758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.955679    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq2xh\" (UniqueName: \"kubernetes.io/projected/0078ab5a-2a09-4107-9e24-e69e5a429c81-kube-api-access-wq2xh\") pod \"0078ab5a-2a09-4107-9e24-e69e5a429c81\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") "
Jul 19 14:25:03.955758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.955734    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0078ab5a-2a09-4107-9e24-e69e5a429c81-webhook-cert\") pod \"0078ab5a-2a09-4107-9e24-e69e5a429c81\" (UID: \"0078ab5a-2a09-4107-9e24-e69e5a429c81\") "
Jul 19 14:25:03.957845 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.957801    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0078ab5a-2a09-4107-9e24-e69e5a429c81-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0078ab5a-2a09-4107-9e24-e69e5a429c81" (UID: "0078ab5a-2a09-4107-9e24-e69e5a429c81"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:25:03.957971 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:03.957909    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0078ab5a-2a09-4107-9e24-e69e5a429c81-kube-api-access-wq2xh" (OuterVolumeSpecName: "kube-api-access-wq2xh") pod "0078ab5a-2a09-4107-9e24-e69e5a429c81" (UID: "0078ab5a-2a09-4107-9e24-e69e5a429c81"). InnerVolumeSpecName "kube-api-access-wq2xh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:25:04.057085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.057059    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq2xh\" (UniqueName: \"kubernetes.io/projected/0078ab5a-2a09-4107-9e24-e69e5a429c81-kube-api-access-wq2xh\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:25:04.057085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.057083    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0078ab5a-2a09-4107-9e24-e69e5a429c81-webhook-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:25:04.718972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.718889    2574 generic.go:356] "Generic (PLEG): container finished" podID="0078ab5a-2a09-4107-9e24-e69e5a429c81" containerID="b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467" exitCode=0
Jul 19 14:25:04.718972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.718931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-bvqn8" event={"ID":"0078ab5a-2a09-4107-9e24-e69e5a429c81","Type":"ContainerDied","Data":"b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467"}
Jul 19 14:25:04.718972 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.718940    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-bvqn8"
Jul 19 14:25:04.719461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.718981    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-bvqn8" event={"ID":"0078ab5a-2a09-4107-9e24-e69e5a429c81","Type":"ContainerDied","Data":"0a23a879a50f64a8104dbf2e56e3049b178a6f28ee173de70f4c70251e7b5901"}
Jul 19 14:25:04.719461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.719004    2574 scope.go:117] "RemoveContainer" containerID="b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467"
Jul 19 14:25:04.727217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.727205    2574 scope.go:117] "RemoveContainer" containerID="b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467"
Jul 19 14:25:04.727440 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:25:04.727419    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467\": container with ID starting with b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467 not found: ID does not exist" containerID="b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467"
Jul 19 14:25:04.727506 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.727448    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467"} err="failed to get container status \"b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467\": rpc error: code = NotFound desc = could not find container \"b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467\": container with ID starting with b0af3b33342fa1d0117acf580c77deebc874c8fc515a6d01f47fc35941331467 not found: ID does not exist"
Jul 19 14:25:04.735802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.735780    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57866856f9-bvqn8"]
Jul 19 14:25:04.741381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:04.741357    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57866856f9-bvqn8"]
Jul 19 14:25:06.396604 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:25:06.396574    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0078ab5a-2a09-4107-9e24-e69e5a429c81" path="/var/lib/kubelet/pods/0078ab5a-2a09-4107-9e24-e69e5a429c81/volumes"
Jul 19 14:26:05.564517 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.564483    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57866856f9-7nxp7"]
Jul 19 14:26:05.564917 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.564875    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0078ab5a-2a09-4107-9e24-e69e5a429c81" containerName="manager"
Jul 19 14:26:05.564917 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.564888    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0078ab5a-2a09-4107-9e24-e69e5a429c81" containerName="manager"
Jul 19 14:26:05.564987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.564952    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0078ab5a-2a09-4107-9e24-e69e5a429c81" containerName="manager"
Jul 19 14:26:05.568991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.568977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.572461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.572437    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2jtlk\""
Jul 19 14:26:05.572461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.572454    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 19 14:26:05.572461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.572461    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 19 14:26:05.577400 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.577380    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57866856f9-7nxp7"]
Jul 19 14:26:05.668294 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.668265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edeee9d0-c244-41a0-bbad-7c7cdc73649f-webhook-cert\") pod \"maas-controller-57866856f9-7nxp7\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") " pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.668442 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.668343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w6pw\" (UniqueName: \"kubernetes.io/projected/edeee9d0-c244-41a0-bbad-7c7cdc73649f-kube-api-access-6w6pw\") pod \"maas-controller-57866856f9-7nxp7\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") " pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.769742 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.769707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w6pw\" (UniqueName: \"kubernetes.io/projected/edeee9d0-c244-41a0-bbad-7c7cdc73649f-kube-api-access-6w6pw\") pod \"maas-controller-57866856f9-7nxp7\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") " pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.769914 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.769803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edeee9d0-c244-41a0-bbad-7c7cdc73649f-webhook-cert\") pod \"maas-controller-57866856f9-7nxp7\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") " pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.772268 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.772244    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edeee9d0-c244-41a0-bbad-7c7cdc73649f-webhook-cert\") pod \"maas-controller-57866856f9-7nxp7\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") " pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.777168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.777148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w6pw\" (UniqueName: \"kubernetes.io/projected/edeee9d0-c244-41a0-bbad-7c7cdc73649f-kube-api-access-6w6pw\") pod \"maas-controller-57866856f9-7nxp7\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") " pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:05.879580 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:05.879550    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:06.004100 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:06.004079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57866856f9-7nxp7"]
Jul 19 14:26:06.005919 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:26:06.005892    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedeee9d0_c244_41a0_bbad_7c7cdc73649f.slice/crio-f91440d43f503479dbf228807956800d3f86a119d354dd9567b747b3bcaa5411 WatchSource:0}: Error finding container f91440d43f503479dbf228807956800d3f86a119d354dd9567b747b3bcaa5411: Status 404 returned error can't find the container with id f91440d43f503479dbf228807956800d3f86a119d354dd9567b747b3bcaa5411
Jul 19 14:26:06.007216 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:06.007198    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:26:07.002391 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:07.002360    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-7nxp7" event={"ID":"edeee9d0-c244-41a0-bbad-7c7cdc73649f","Type":"ContainerStarted","Data":"ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e"}
Jul 19 14:26:07.002786 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:07.002406    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-7nxp7" event={"ID":"edeee9d0-c244-41a0-bbad-7c7cdc73649f","Type":"ContainerStarted","Data":"f91440d43f503479dbf228807956800d3f86a119d354dd9567b747b3bcaa5411"}
Jul 19 14:26:07.002786 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:07.002489    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:07.019250 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:07.019206    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57866856f9-7nxp7" podStartSLOduration=1.626719292 podStartE2EDuration="2.019191735s" totalImagesPullingTime="392.472443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:26:07.017904398 +0000 UTC m=+785.320735416" watchObservedRunningTime="2026-07-19 14:26:07.019191735 +0000 UTC m=+785.322022764"
Jul 19 14:26:18.009406 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:18.009377    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:22.587959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:22.587930    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:26:22.663487 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:22.663449    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57866856f9-7nxp7"]
Jul 19 14:26:22.663663 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:22.663644    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57866856f9-7nxp7" podUID="edeee9d0-c244-41a0-bbad-7c7cdc73649f" containerName="manager" containerID="cri-o://ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e" gracePeriod=30
Jul 19 14:26:22.907356 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:22.907327    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:23.024939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.024906    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edeee9d0-c244-41a0-bbad-7c7cdc73649f-webhook-cert\") pod \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") "
Jul 19 14:26:23.025102 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.024961    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w6pw\" (UniqueName: \"kubernetes.io/projected/edeee9d0-c244-41a0-bbad-7c7cdc73649f-kube-api-access-6w6pw\") pod \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\" (UID: \"edeee9d0-c244-41a0-bbad-7c7cdc73649f\") "
Jul 19 14:26:23.026870 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.026844    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edeee9d0-c244-41a0-bbad-7c7cdc73649f-kube-api-access-6w6pw" (OuterVolumeSpecName: "kube-api-access-6w6pw") pod "edeee9d0-c244-41a0-bbad-7c7cdc73649f" (UID: "edeee9d0-c244-41a0-bbad-7c7cdc73649f"). InnerVolumeSpecName "kube-api-access-6w6pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:26:23.026982 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.026863    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edeee9d0-c244-41a0-bbad-7c7cdc73649f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "edeee9d0-c244-41a0-bbad-7c7cdc73649f" (UID: "edeee9d0-c244-41a0-bbad-7c7cdc73649f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:26:23.076964 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.076933    2574 generic.go:356] "Generic (PLEG): container finished" podID="edeee9d0-c244-41a0-bbad-7c7cdc73649f" containerID="ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e" exitCode=0
Jul 19 14:26:23.077106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.076973    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-7nxp7"
Jul 19 14:26:23.077106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.077022    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-7nxp7" event={"ID":"edeee9d0-c244-41a0-bbad-7c7cdc73649f","Type":"ContainerDied","Data":"ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e"}
Jul 19 14:26:23.077106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.077066    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-7nxp7" event={"ID":"edeee9d0-c244-41a0-bbad-7c7cdc73649f","Type":"ContainerDied","Data":"f91440d43f503479dbf228807956800d3f86a119d354dd9567b747b3bcaa5411"}
Jul 19 14:26:23.077106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.077086    2574 scope.go:117] "RemoveContainer" containerID="ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e"
Jul 19 14:26:23.085484 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.085469    2574 scope.go:117] "RemoveContainer" containerID="ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e"
Jul 19 14:26:23.085752 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:26:23.085735    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e\": container with ID starting with ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e not found: ID does not exist" containerID="ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e"
Jul 19 14:26:23.085808 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.085760    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e"} err="failed to get container status \"ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e\": rpc error: code = NotFound desc = could not find container \"ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e\": container with ID starting with ef2f1244cbdb38b074b2b08cc37912a12a83e7947d09a487b06faf5f9896da4e not found: ID does not exist"
Jul 19 14:26:23.097738 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.097707    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57866856f9-7nxp7"]
Jul 19 14:26:23.098986 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.098967    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57866856f9-7nxp7"]
Jul 19 14:26:23.126155 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.126134    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w6pw\" (UniqueName: \"kubernetes.io/projected/edeee9d0-c244-41a0-bbad-7c7cdc73649f-kube-api-access-6w6pw\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:26:23.126155 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.126156    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edeee9d0-c244-41a0-bbad-7c7cdc73649f-webhook-cert\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:26:23.765834 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.765799    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:26:23.887288 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.887259    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57866856f9-ztrk5"]
Jul 19 14:26:23.887648 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.887635    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edeee9d0-c244-41a0-bbad-7c7cdc73649f" containerName="manager"
Jul 19 14:26:23.887714 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.887650    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="edeee9d0-c244-41a0-bbad-7c7cdc73649f" containerName="manager"
Jul 19 14:26:23.887754 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.887722    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="edeee9d0-c244-41a0-bbad-7c7cdc73649f" containerName="manager"
Jul 19 14:26:23.891912 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.891896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:23.895272 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.895240    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2jtlk\""
Jul 19 14:26:23.895378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.895287    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 19 14:26:23.895378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.895296    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 19 14:26:23.899420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:23.899400    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57866856f9-ztrk5"]
Jul 19 14:26:24.032081 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.032000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n28pb\" (UniqueName: \"kubernetes.io/projected/3a37e812-ea38-4904-b2b1-995cfac53432-kube-api-access-n28pb\") pod \"maas-controller-57866856f9-ztrk5\" (UID: \"3a37e812-ea38-4904-b2b1-995cfac53432\") " pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.032081 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.032052    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a37e812-ea38-4904-b2b1-995cfac53432-webhook-cert\") pod \"maas-controller-57866856f9-ztrk5\" (UID: \"3a37e812-ea38-4904-b2b1-995cfac53432\") " pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.133414 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.133380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28pb\" (UniqueName: \"kubernetes.io/projected/3a37e812-ea38-4904-b2b1-995cfac53432-kube-api-access-n28pb\") pod \"maas-controller-57866856f9-ztrk5\" (UID: \"3a37e812-ea38-4904-b2b1-995cfac53432\") " pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.133622 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.133436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a37e812-ea38-4904-b2b1-995cfac53432-webhook-cert\") pod \"maas-controller-57866856f9-ztrk5\" (UID: \"3a37e812-ea38-4904-b2b1-995cfac53432\") " pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.135782 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.135754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a37e812-ea38-4904-b2b1-995cfac53432-webhook-cert\") pod \"maas-controller-57866856f9-ztrk5\" (UID: \"3a37e812-ea38-4904-b2b1-995cfac53432\") " pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.140819 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.140796    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n28pb\" (UniqueName: \"kubernetes.io/projected/3a37e812-ea38-4904-b2b1-995cfac53432-kube-api-access-n28pb\") pod \"maas-controller-57866856f9-ztrk5\" (UID: \"3a37e812-ea38-4904-b2b1-995cfac53432\") " pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.202491 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.202425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:24.329708 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.329682    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57866856f9-ztrk5"]
Jul 19 14:26:24.332068 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:26:24.332042    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a37e812_ea38_4904_b2b1_995cfac53432.slice/crio-f0a51eab30717544efeff3fa509560fe8177cc469f63b825e193d8877f35e2d3 WatchSource:0}: Error finding container f0a51eab30717544efeff3fa509560fe8177cc469f63b825e193d8877f35e2d3: Status 404 returned error can't find the container with id f0a51eab30717544efeff3fa509560fe8177cc469f63b825e193d8877f35e2d3
Jul 19 14:26:24.397262 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:24.397229    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edeee9d0-c244-41a0-bbad-7c7cdc73649f" path="/var/lib/kubelet/pods/edeee9d0-c244-41a0-bbad-7c7cdc73649f/volumes"
Jul 19 14:26:25.088583 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:25.088498    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-ztrk5" event={"ID":"3a37e812-ea38-4904-b2b1-995cfac53432","Type":"ContainerStarted","Data":"06e50b0968eb3bd7c4c1e67eeb4176bbda8c47ee2475b5a407b2319b9c03378e"}
Jul 19 14:26:25.088583 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:25.088558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57866856f9-ztrk5" event={"ID":"3a37e812-ea38-4904-b2b1-995cfac53432","Type":"ContainerStarted","Data":"f0a51eab30717544efeff3fa509560fe8177cc469f63b825e193d8877f35e2d3"}
Jul 19 14:26:25.088951 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:25.088588    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:25.105307 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:25.105261    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57866856f9-ztrk5" podStartSLOduration=1.804499148 podStartE2EDuration="2.105245617s" totalImagesPullingTime="300.746469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:26:25.103949758 +0000 UTC m=+803.406780775" watchObservedRunningTime="2026-07-19 14:26:25.105245617 +0000 UTC m=+803.408076633"
Jul 19 14:26:36.095671 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:36.095638    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57866856f9-ztrk5"
Jul 19 14:26:59.381802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:59.381763    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:26:59.481881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:59.481846    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:26:59.598960 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:26:59.598920    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:05.214491 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:05.214458    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:05.275758 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:05.275722    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:05.354737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:05.354706    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:05.453322 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:05.453292    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:05.564288 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:05.564201    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:11.339748 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:11.339707    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:27:11.411759 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:27:11.411725    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:28:02.086935 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:28:02.086862    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:28:02.089694 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:28:02.089670    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:30:25.974833 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:30:25.974801    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:30:50.627244 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:30:50.627209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:31:05.812397 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:31:05.812320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:32:59.155783 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:32:59.155752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:33:02.114748 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:33:02.114710    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:33:02.117517 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:33:02.117483    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:34:39.218510 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:34:39.218475    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:34:48.118404 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:34:48.118368    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:35:20.621213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:35:20.621166    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:35:21.401369 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:35:21.401335    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:02.140795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:02.140766    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:38:02.144303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:02.144277    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:38:31.515249 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:31.515169    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:32.706982 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:32.706940    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:37.056887 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:37.056845    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:37.115988 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:37.115957    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:37.298136 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:37.298093    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:37.609359 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:37.609330    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:37.911721 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:37.911638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:39.967878 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:39.967845    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:40.067467 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.067432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:40.213003 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.212971    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"]
Jul 19 14:38:40.216636 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.216607    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.219303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.219247    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2e6251c8-serving-cert\""
Jul 19 14:38:40.227451 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.227431    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"]
Jul 19 14:38:40.247856 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.247782    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:40.269870 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.269849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.270002 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.269934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f6h8\" (UniqueName: \"kubernetes.io/projected/5fe2769e-e06a-4025-88b2-8f6de06400fa-kube-api-access-2f6h8\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.270002 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.269969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe2769e-e06a-4025-88b2-8f6de06400fa-openshift-service-ca\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.370454 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.370428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe2769e-e06a-4025-88b2-8f6de06400fa-openshift-service-ca\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.370619 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.370475    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.370619 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.370565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f6h8\" (UniqueName: \"kubernetes.io/projected/5fe2769e-e06a-4025-88b2-8f6de06400fa-kube-api-access-2f6h8\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.370695 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:38:40.370649    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-serving-cert: secret "maas-api-e2e-ait-2e6251c8-serving-cert" not found
Jul 19 14:38:40.370737 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:38:40.370709    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls podName:5fe2769e-e06a-4025-88b2-8f6de06400fa nodeName:}" failed. No retries permitted until 2026-07-19 14:38:40.870689411 +0000 UTC m=+1539.173520406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls") pod "maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" (UID: "5fe2769e-e06a-4025-88b2-8f6de06400fa") : secret "maas-api-e2e-ait-2e6251c8-serving-cert" not found
Jul 19 14:38:40.371202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.371182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe2769e-e06a-4025-88b2-8f6de06400fa-openshift-service-ca\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.383103 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.383079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f6h8\" (UniqueName: \"kubernetes.io/projected/5fe2769e-e06a-4025-88b2-8f6de06400fa-kube-api-access-2f6h8\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.731997 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.731964    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:40.875232 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.875196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.877595 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.877576    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls\") pod \"maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:40.899026 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.898994    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:40.993864 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:40.993786    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:41.128584 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:41.128551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:41.264508 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:41.264435    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"]
Jul 19 14:38:41.267613 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:38:41.267585    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe2769e_e06a_4025_88b2_8f6de06400fa.slice/crio-9087c2f1d8bd9a282a2ef3c3feed9a7be607f162476c70122eef5d978b4e78bb WatchSource:0}: Error finding container 9087c2f1d8bd9a282a2ef3c3feed9a7be607f162476c70122eef5d978b4e78bb: Status 404 returned error can't find the container with id 9087c2f1d8bd9a282a2ef3c3feed9a7be607f162476c70122eef5d978b4e78bb
Jul 19 14:38:41.269075 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:41.269052    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:38:41.357459 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:41.357425    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" event={"ID":"5fe2769e-e06a-4025-88b2-8f6de06400fa","Type":"ContainerStarted","Data":"9087c2f1d8bd9a282a2ef3c3feed9a7be607f162476c70122eef5d978b4e78bb"}
Jul 19 14:38:42.363043 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:42.363008    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" event={"ID":"5fe2769e-e06a-4025-88b2-8f6de06400fa","Type":"ContainerStarted","Data":"c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3"}
Jul 19 14:38:42.363443 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:42.363094    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:38:42.379293 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:42.379247    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" podStartSLOduration=1.869888255 podStartE2EDuration="2.379234048s" totalImagesPullingTime="509.345793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:38:42.37791457 +0000 UTC m=+1540.680745612" watchObservedRunningTime="2026-07-19 14:38:42.379234048 +0000 UTC m=+1540.682065054"
Jul 19 14:38:48.372552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:38:48.372506    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:39:04.293849 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:04.293816    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:15.911363 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:15.911330    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"]
Jul 19 14:39:15.911932 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:15.911596    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" podUID="5fe2769e-e06a-4025-88b2-8f6de06400fa" containerName="maas-api" containerID="cri-o://c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3" gracePeriod=30
Jul 19 14:39:15.938577 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:15.938551    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:16.159028 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.159003    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:39:16.280337 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.280264    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls\") pod \"5fe2769e-e06a-4025-88b2-8f6de06400fa\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") "
Jul 19 14:39:16.280467 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.280355    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f6h8\" (UniqueName: \"kubernetes.io/projected/5fe2769e-e06a-4025-88b2-8f6de06400fa-kube-api-access-2f6h8\") pod \"5fe2769e-e06a-4025-88b2-8f6de06400fa\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") "
Jul 19 14:39:16.280467 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.280424    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe2769e-e06a-4025-88b2-8f6de06400fa-openshift-service-ca\") pod \"5fe2769e-e06a-4025-88b2-8f6de06400fa\" (UID: \"5fe2769e-e06a-4025-88b2-8f6de06400fa\") "
Jul 19 14:39:16.280766 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.280744    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe2769e-e06a-4025-88b2-8f6de06400fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fe2769e-e06a-4025-88b2-8f6de06400fa" (UID: "5fe2769e-e06a-4025-88b2-8f6de06400fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:39:16.282465 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.282444    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe2769e-e06a-4025-88b2-8f6de06400fa-kube-api-access-2f6h8" (OuterVolumeSpecName: "kube-api-access-2f6h8") pod "5fe2769e-e06a-4025-88b2-8f6de06400fa" (UID: "5fe2769e-e06a-4025-88b2-8f6de06400fa"). InnerVolumeSpecName "kube-api-access-2f6h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:39:16.282548 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.282452    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fe2769e-e06a-4025-88b2-8f6de06400fa" (UID: "5fe2769e-e06a-4025-88b2-8f6de06400fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:39:16.381678 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.381647    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe2769e-e06a-4025-88b2-8f6de06400fa-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:39:16.381678 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.381673    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe2769e-e06a-4025-88b2-8f6de06400fa-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:39:16.381821 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.381688    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f6h8\" (UniqueName: \"kubernetes.io/projected/5fe2769e-e06a-4025-88b2-8f6de06400fa-kube-api-access-2f6h8\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:39:16.512146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.512114    2574 generic.go:356] "Generic (PLEG): container finished" podID="5fe2769e-e06a-4025-88b2-8f6de06400fa" containerID="c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3" exitCode=0
Jul 19 14:39:16.512378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.512161    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" event={"ID":"5fe2769e-e06a-4025-88b2-8f6de06400fa","Type":"ContainerDied","Data":"c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3"}
Jul 19 14:39:16.512378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.512162    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"
Jul 19 14:39:16.512378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.512200    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2" event={"ID":"5fe2769e-e06a-4025-88b2-8f6de06400fa","Type":"ContainerDied","Data":"9087c2f1d8bd9a282a2ef3c3feed9a7be607f162476c70122eef5d978b4e78bb"}
Jul 19 14:39:16.512378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.512220    2574 scope.go:117] "RemoveContainer" containerID="c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3"
Jul 19 14:39:16.520588 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.520567    2574 scope.go:117] "RemoveContainer" containerID="c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3"
Jul 19 14:39:16.520821 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:39:16.520804    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3\": container with ID starting with c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3 not found: ID does not exist" containerID="c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3"
Jul 19 14:39:16.520879 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.520828    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3"} err="failed to get container status \"c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3\": rpc error: code = NotFound desc = could not find container \"c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3\": container with ID starting with c4223f6b68413467772413d8fbd988c3fc9fed41cce55f7911f0dfd776f0f8e3 not found: ID does not exist"
Jul 19 14:39:16.529182 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.529158    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"]
Jul 19 14:39:16.530765 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:16.530721    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e6251c8-5c56946c67-2xss2"]
Jul 19 14:39:18.397111 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:18.397078    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe2769e-e06a-4025-88b2-8f6de06400fa" path="/var/lib/kubelet/pods/5fe2769e-e06a-4025-88b2-8f6de06400fa/volumes"
Jul 19 14:39:20.603072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:20.603037    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:20.761323 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:20.761285    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:20.993997 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:20.993961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:21.242160 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:21.242128    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:21.339703 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:21.339621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:21.455655 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:21.455621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:21.791457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:21.791403    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:22.376278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:22.376247    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:24.392362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.392327    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:24.491437 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.491406    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"]
Jul 19 14:39:24.491934 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.491915    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe2769e-e06a-4025-88b2-8f6de06400fa" containerName="maas-api"
Jul 19 14:39:24.491934 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.491934    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe2769e-e06a-4025-88b2-8f6de06400fa" containerName="maas-api"
Jul 19 14:39:24.492090 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.491991    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe2769e-e06a-4025-88b2-8f6de06400fa" containerName="maas-api"
Jul 19 14:39:24.496344 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.496324    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.498468 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.498451    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c6a7edc8-serving-cert\""
Jul 19 14:39:24.505663 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.505627    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"]
Jul 19 14:39:24.511288 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.511254    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:24.611664 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.611630    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:24.654872 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.654800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl6z4\" (UniqueName: \"kubernetes.io/projected/7a936353-7a8a-4aee-8e64-4403ed3eae7a-kube-api-access-tl6z4\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.654872 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.654837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a936353-7a8a-4aee-8e64-4403ed3eae7a-openshift-service-ca\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.655029 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.654884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a936353-7a8a-4aee-8e64-4403ed3eae7a-maas-api-tls\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.755981 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.755952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl6z4\" (UniqueName: \"kubernetes.io/projected/7a936353-7a8a-4aee-8e64-4403ed3eae7a-kube-api-access-tl6z4\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.755981 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.755987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a936353-7a8a-4aee-8e64-4403ed3eae7a-openshift-service-ca\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.756230 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.756012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a936353-7a8a-4aee-8e64-4403ed3eae7a-maas-api-tls\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.756684 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.756662    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a936353-7a8a-4aee-8e64-4403ed3eae7a-openshift-service-ca\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.758501 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.758476    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a936353-7a8a-4aee-8e64-4403ed3eae7a-maas-api-tls\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.763692 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.763663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl6z4\" (UniqueName: \"kubernetes.io/projected/7a936353-7a8a-4aee-8e64-4403ed3eae7a-kube-api-access-tl6z4\") pod \"maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.807465 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.807418    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:24.943602 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.943576    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"]
Jul 19 14:39:24.946301 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:39:24.946275    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a936353_7a8a_4aee_8e64_4403ed3eae7a.slice/crio-7a80ad53367c2223935cb41115f8b5d0d90020142df92158a2590e50584a81af WatchSource:0}: Error finding container 7a80ad53367c2223935cb41115f8b5d0d90020142df92158a2590e50584a81af: Status 404 returned error can't find the container with id 7a80ad53367c2223935cb41115f8b5d0d90020142df92158a2590e50584a81af
Jul 19 14:39:24.964207 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:24.964175    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:25.066019 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:25.065990    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:25.554369 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:25.554332    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2" event={"ID":"7a936353-7a8a-4aee-8e64-4403ed3eae7a","Type":"ContainerStarted","Data":"13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09"}
Jul 19 14:39:25.554805 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:25.554384    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2" event={"ID":"7a936353-7a8a-4aee-8e64-4403ed3eae7a","Type":"ContainerStarted","Data":"7a80ad53367c2223935cb41115f8b5d0d90020142df92158a2590e50584a81af"}
Jul 19 14:39:25.554805 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:25.554452    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:25.571306 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:25.571218    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2" podStartSLOduration=1.212827772 podStartE2EDuration="1.571201878s" totalImagesPullingTime="358.374106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:39:25.569394332 +0000 UTC m=+1583.872225348" watchObservedRunningTime="2026-07-19 14:39:25.571201878 +0000 UTC m=+1583.874032894"
Jul 19 14:39:31.563490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:31.563465    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:39:59.784601 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:59.784505    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"]
Jul 19 14:39:59.785114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:59.784783    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2" podUID="7a936353-7a8a-4aee-8e64-4403ed3eae7a" containerName="maas-api" containerID="cri-o://13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09" gracePeriod=30
Jul 19 14:39:59.802113 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:39:59.802087    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:00.025248 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.025223    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:40:00.149800 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.149765    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl6z4\" (UniqueName: \"kubernetes.io/projected/7a936353-7a8a-4aee-8e64-4403ed3eae7a-kube-api-access-tl6z4\") pod \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") "
Jul 19 14:40:00.149953 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.149820    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a936353-7a8a-4aee-8e64-4403ed3eae7a-maas-api-tls\") pod \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") "
Jul 19 14:40:00.149953 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.149927    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a936353-7a8a-4aee-8e64-4403ed3eae7a-openshift-service-ca\") pod \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\" (UID: \"7a936353-7a8a-4aee-8e64-4403ed3eae7a\") "
Jul 19 14:40:00.150241 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.150216    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a936353-7a8a-4aee-8e64-4403ed3eae7a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a936353-7a8a-4aee-8e64-4403ed3eae7a" (UID: "7a936353-7a8a-4aee-8e64-4403ed3eae7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:40:00.151762 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.151744    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a936353-7a8a-4aee-8e64-4403ed3eae7a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a936353-7a8a-4aee-8e64-4403ed3eae7a" (UID: "7a936353-7a8a-4aee-8e64-4403ed3eae7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:40:00.151834 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.151753    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a936353-7a8a-4aee-8e64-4403ed3eae7a-kube-api-access-tl6z4" (OuterVolumeSpecName: "kube-api-access-tl6z4") pod "7a936353-7a8a-4aee-8e64-4403ed3eae7a" (UID: "7a936353-7a8a-4aee-8e64-4403ed3eae7a"). InnerVolumeSpecName "kube-api-access-tl6z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:40:00.251230 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.251206    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a936353-7a8a-4aee-8e64-4403ed3eae7a-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:40:00.251230 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.251228    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl6z4\" (UniqueName: \"kubernetes.io/projected/7a936353-7a8a-4aee-8e64-4403ed3eae7a-kube-api-access-tl6z4\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:40:00.251230 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.251237    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a936353-7a8a-4aee-8e64-4403ed3eae7a-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:40:00.705508 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.705434    2574 generic.go:356] "Generic (PLEG): container finished" podID="7a936353-7a8a-4aee-8e64-4403ed3eae7a" containerID="13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09" exitCode=0
Jul 19 14:40:00.705508 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.705476    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"
Jul 19 14:40:00.705700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.705514    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2" event={"ID":"7a936353-7a8a-4aee-8e64-4403ed3eae7a","Type":"ContainerDied","Data":"13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09"}
Jul 19 14:40:00.705700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.705587    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2" event={"ID":"7a936353-7a8a-4aee-8e64-4403ed3eae7a","Type":"ContainerDied","Data":"7a80ad53367c2223935cb41115f8b5d0d90020142df92158a2590e50584a81af"}
Jul 19 14:40:00.705700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.705608    2574 scope.go:117] "RemoveContainer" containerID="13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09"
Jul 19 14:40:00.713396 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.713380    2574 scope.go:117] "RemoveContainer" containerID="13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09"
Jul 19 14:40:00.713683 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:00.713660    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09\": container with ID starting with 13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09 not found: ID does not exist" containerID="13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09"
Jul 19 14:40:00.713769 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.713688    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09"} err="failed to get container status \"13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09\": rpc error: code = NotFound desc = could not find container \"13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09\": container with ID starting with 13a563f2e336b1d883e774cb54027574e779e1f73ee35be031ebfc78a70c2b09 not found: ID does not exist"
Jul 19 14:40:00.720942 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.720923    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"]
Jul 19 14:40:00.725686 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:00.725660    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6a7edc8-7f4bccd77c-cfmf2"]
Jul 19 14:40:02.398567 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:02.398513    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a936353-7a8a-4aee-8e64-4403ed3eae7a" path="/var/lib/kubelet/pods/7a936353-7a8a-4aee-8e64-4403ed3eae7a/volumes"
Jul 19 14:40:04.379956 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:04.379918    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:04.548403 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:04.548353    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:04.787267 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:04.787182    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:05.009165 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:05.009132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:05.334333 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:05.334291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:05.493795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:05.493764    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:07.420588 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.420505    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:07.546959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.546929    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"]
Jul 19 14:40:07.547304 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.547286    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a936353-7a8a-4aee-8e64-4403ed3eae7a" containerName="maas-api"
Jul 19 14:40:07.547304 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.547301    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a936353-7a8a-4aee-8e64-4403ed3eae7a" containerName="maas-api"
Jul 19 14:40:07.547479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.547371    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a936353-7a8a-4aee-8e64-4403ed3eae7a" containerName="maas-api"
Jul 19 14:40:07.550319 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.550296    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.550624 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.550598    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:07.553188 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.553169    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-25213646-serving-cert\""
Jul 19 14:40:07.613242 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.613208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edb17e6-1ddb-4935-8f25-45c756c1ad85-openshift-service-ca\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.613381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.613273    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89p6r\" (UniqueName: \"kubernetes.io/projected/1edb17e6-1ddb-4935-8f25-45c756c1ad85-kube-api-access-89p6r\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.613381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.613345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.629786 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.629760    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"]
Jul 19 14:40:07.673644 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.673563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:07.714011 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.713982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89p6r\" (UniqueName: \"kubernetes.io/projected/1edb17e6-1ddb-4935-8f25-45c756c1ad85-kube-api-access-89p6r\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.714136 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.714029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.714136 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.714071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edb17e6-1ddb-4935-8f25-45c756c1ad85-openshift-service-ca\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.714223 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:07.714165    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-25213646-serving-cert: secret "maas-api-e2e-ait-25213646-serving-cert" not found
Jul 19 14:40:07.714259 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:07.714223    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls podName:1edb17e6-1ddb-4935-8f25-45c756c1ad85 nodeName:}" failed. No retries permitted until 2026-07-19 14:40:08.214206518 +0000 UTC m=+1626.517037526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls") pod "maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" (UID: "1edb17e6-1ddb-4935-8f25-45c756c1ad85") : secret "maas-api-e2e-ait-25213646-serving-cert" not found
Jul 19 14:40:07.714644 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.714629    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edb17e6-1ddb-4935-8f25-45c756c1ad85-openshift-service-ca\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.724894 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.724869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89p6r\" (UniqueName: \"kubernetes.io/projected/1edb17e6-1ddb-4935-8f25-45c756c1ad85-kube-api-access-89p6r\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:07.966457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:07.966381    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:08.130907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.130872    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:08.217901 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.217825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:08.220176 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.220151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls\") pod \"maas-api-e2e-ait-25213646-6cf99d95cb-rvscw\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:08.233349 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.233322    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:08.460847 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.460816    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:08.593765 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.593744    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"]
Jul 19 14:40:08.595778 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:40:08.595742    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edb17e6_1ddb_4935_8f25_45c756c1ad85.slice/crio-3b592064941d9902add069ac5a6f949fd728fd4bd3fee2eb48bc4dae68f34f73 WatchSource:0}: Error finding container 3b592064941d9902add069ac5a6f949fd728fd4bd3fee2eb48bc4dae68f34f73: Status 404 returned error can't find the container with id 3b592064941d9902add069ac5a6f949fd728fd4bd3fee2eb48bc4dae68f34f73
Jul 19 14:40:08.745246 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:08.745155    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" event={"ID":"1edb17e6-1ddb-4935-8f25-45c756c1ad85","Type":"ContainerStarted","Data":"3b592064941d9902add069ac5a6f949fd728fd4bd3fee2eb48bc4dae68f34f73"}
Jul 19 14:40:09.751840 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:09.751792    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" event={"ID":"1edb17e6-1ddb-4935-8f25-45c756c1ad85","Type":"ContainerStarted","Data":"b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de"}
Jul 19 14:40:09.752217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:09.751863    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:09.767485 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:09.767438    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" podStartSLOduration=2.261268956 podStartE2EDuration="2.767420348s" totalImagesPullingTime="506.151392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:40:09.765810605 +0000 UTC m=+1628.068641620" watchObservedRunningTime="2026-07-19 14:40:09.767420348 +0000 UTC m=+1628.070251363"
Jul 19 14:40:15.762075 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:15.762044    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:43.627065 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.627028    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"]
Jul 19 14:40:43.630137 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.627778    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" podUID="1edb17e6-1ddb-4935-8f25-45c756c1ad85" containerName="maas-api" containerID="cri-o://b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de" gracePeriod=30
Jul 19 14:40:43.658102 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.658049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:43.871538 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.871504    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:43.905131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.905068    2574 generic.go:356] "Generic (PLEG): container finished" podID="1edb17e6-1ddb-4935-8f25-45c756c1ad85" containerID="b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de" exitCode=0
Jul 19 14:40:43.905131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.905112    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"
Jul 19 14:40:43.905304 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.905153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" event={"ID":"1edb17e6-1ddb-4935-8f25-45c756c1ad85","Type":"ContainerDied","Data":"b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de"}
Jul 19 14:40:43.905304 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.905240    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw" event={"ID":"1edb17e6-1ddb-4935-8f25-45c756c1ad85","Type":"ContainerDied","Data":"3b592064941d9902add069ac5a6f949fd728fd4bd3fee2eb48bc4dae68f34f73"}
Jul 19 14:40:43.905304 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.905277    2574 scope.go:117] "RemoveContainer" containerID="b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de"
Jul 19 14:40:43.913180 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.913152    2574 scope.go:117] "RemoveContainer" containerID="b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de"
Jul 19 14:40:43.913410 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:43.913392    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de\": container with ID starting with b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de not found: ID does not exist" containerID="b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de"
Jul 19 14:40:43.913471 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.913418    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de"} err="failed to get container status \"b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de\": rpc error: code = NotFound desc = could not find container \"b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de\": container with ID starting with b8f690a25d922cf1e7cd1ebcb137275316f83346c40215c770599a7c93e3a2de not found: ID does not exist"
Jul 19 14:40:43.926300 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.926279    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edb17e6-1ddb-4935-8f25-45c756c1ad85-openshift-service-ca\") pod \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") "
Jul 19 14:40:43.926380 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.926331    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89p6r\" (UniqueName: \"kubernetes.io/projected/1edb17e6-1ddb-4935-8f25-45c756c1ad85-kube-api-access-89p6r\") pod \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") "
Jul 19 14:40:43.926380 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.926367    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls\") pod \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\" (UID: \"1edb17e6-1ddb-4935-8f25-45c756c1ad85\") "
Jul 19 14:40:43.926636 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.926615    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edb17e6-1ddb-4935-8f25-45c756c1ad85-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1edb17e6-1ddb-4935-8f25-45c756c1ad85" (UID: "1edb17e6-1ddb-4935-8f25-45c756c1ad85"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:40:43.928179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.928158    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edb17e6-1ddb-4935-8f25-45c756c1ad85-kube-api-access-89p6r" (OuterVolumeSpecName: "kube-api-access-89p6r") pod "1edb17e6-1ddb-4935-8f25-45c756c1ad85" (UID: "1edb17e6-1ddb-4935-8f25-45c756c1ad85"). InnerVolumeSpecName "kube-api-access-89p6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:40:43.928262 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:43.928207    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1edb17e6-1ddb-4935-8f25-45c756c1ad85" (UID: "1edb17e6-1ddb-4935-8f25-45c756c1ad85"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:40:44.027557 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:44.027511    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89p6r\" (UniqueName: \"kubernetes.io/projected/1edb17e6-1ddb-4935-8f25-45c756c1ad85-kube-api-access-89p6r\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:40:44.027557 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:44.027555    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1edb17e6-1ddb-4935-8f25-45c756c1ad85-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:40:44.027711 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:44.027564    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edb17e6-1ddb-4935-8f25-45c756c1ad85-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:40:44.225874 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:44.225844    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"]
Jul 19 14:40:44.227810 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:44.227788    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25213646-6cf99d95cb-rvscw"]
Jul 19 14:40:44.396660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:44.396632    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edb17e6-1ddb-4935-8f25-45c756c1ad85" path="/var/lib/kubelet/pods/1edb17e6-1ddb-4935-8f25-45c756c1ad85/volumes"
Jul 19 14:40:48.333056 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:48.333020    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:49.536435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:49.536395    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:49.758101 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:49.758065    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:49.974921 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:49.974885    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:50.292012 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:50.291921    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:50.560366 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:50.560278    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:52.658370 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.658332    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:52.730663 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.730631    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"]
Jul 19 14:40:52.731086 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.731057    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edb17e6-1ddb-4935-8f25-45c756c1ad85" containerName="maas-api"
Jul 19 14:40:52.731266 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.731251    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edb17e6-1ddb-4935-8f25-45c756c1ad85" containerName="maas-api"
Jul 19 14:40:52.731508 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.731490    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edb17e6-1ddb-4935-8f25-45c756c1ad85" containerName="maas-api"
Jul 19 14:40:52.734748 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.734723    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:52.736037 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.736019    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.738235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.738216    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-35312369-serving-cert\""
Jul 19 14:40:52.745288 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.745269    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"]
Jul 19 14:40:52.796992 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.796958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:52.800003 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.799983    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.800103 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.800013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58988304-b2d1-4f0b-80eb-7d8144edc73b-openshift-service-ca\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.800103 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.800087    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs7nt\" (UniqueName: \"kubernetes.io/projected/58988304-b2d1-4f0b-80eb-7d8144edc73b-kube-api-access-vs7nt\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.901103 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.901075    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.901231 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.901108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58988304-b2d1-4f0b-80eb-7d8144edc73b-openshift-service-ca\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.901296 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:52.901232    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-35312369-serving-cert: secret "maas-api-e2e-derive-35312369-serving-cert" not found
Jul 19 14:40:52.901296 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.901248    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs7nt\" (UniqueName: \"kubernetes.io/projected/58988304-b2d1-4f0b-80eb-7d8144edc73b-kube-api-access-vs7nt\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.901385 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:52.901294    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls podName:58988304-b2d1-4f0b-80eb-7d8144edc73b nodeName:}" failed. No retries permitted until 2026-07-19 14:40:53.401272955 +0000 UTC m=+1671.704103953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls") pod "maas-api-e2e-derive-35312369-54f947d48b-fdgrn" (UID: "58988304-b2d1-4f0b-80eb-7d8144edc73b") : secret "maas-api-e2e-derive-35312369-serving-cert" not found
Jul 19 14:40:52.901879 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.901861    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58988304-b2d1-4f0b-80eb-7d8144edc73b-openshift-service-ca\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:52.909347 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:52.909289    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs7nt\" (UniqueName: \"kubernetes.io/projected/58988304-b2d1-4f0b-80eb-7d8144edc73b-kube-api-access-vs7nt\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:53.304203 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:53.304108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:53.404478 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:53.404445    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:53.405496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:53.405474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:53.405610 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:53.405596    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-35312369-serving-cert: secret "maas-api-e2e-derive-35312369-serving-cert" not found
Jul 19 14:40:53.405651 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:40:53.405646    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls podName:58988304-b2d1-4f0b-80eb-7d8144edc73b nodeName:}" failed. No retries permitted until 2026-07-19 14:40:54.405633394 +0000 UTC m=+1672.708464389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls") pod "maas-api-e2e-derive-35312369-54f947d48b-fdgrn" (UID: "58988304-b2d1-4f0b-80eb-7d8144edc73b") : secret "maas-api-e2e-derive-35312369-serving-cert" not found
Jul 19 14:40:53.525798 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:53.525758    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:54.415509 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:54.415475    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:54.417824 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:54.417806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") pod \"maas-api-e2e-derive-35312369-54f947d48b-fdgrn\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:54.548937 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:54.548896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:54.689289 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:54.689203    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"]
Jul 19 14:40:54.693355 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:40:54.693328    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58988304_b2d1_4f0b_80eb_7d8144edc73b.slice/crio-1272403541b83593c06ec36c7d10981920348f7a62dbf8a18fc4ba889f624fc1 WatchSource:0}: Error finding container 1272403541b83593c06ec36c7d10981920348f7a62dbf8a18fc4ba889f624fc1: Status 404 returned error can't find the container with id 1272403541b83593c06ec36c7d10981920348f7a62dbf8a18fc4ba889f624fc1
Jul 19 14:40:54.955356 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:54.955275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn" event={"ID":"58988304-b2d1-4f0b-80eb-7d8144edc73b","Type":"ContainerStarted","Data":"1272403541b83593c06ec36c7d10981920348f7a62dbf8a18fc4ba889f624fc1"}
Jul 19 14:40:55.961416 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:55.961385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn" event={"ID":"58988304-b2d1-4f0b-80eb-7d8144edc73b","Type":"ContainerStarted","Data":"3195909aa6dd672db81ceebff11d24c11afb964b80cc55a97001b9ddef772261"}
Jul 19 14:40:55.961819 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:55.961479    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:40:55.978204 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:55.978158    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn" podStartSLOduration=3.608093254 podStartE2EDuration="3.978141714s" totalImagesPullingTime="370.04846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:40:55.975960504 +0000 UTC m=+1674.278791520" watchObservedRunningTime="2026-07-19 14:40:55.978141714 +0000 UTC m=+1674.280972730"
Jul 19 14:40:56.749377 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:56.749308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:56.848225 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:40:56.848179    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:01.971962 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:01.971935    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:41:27.918014 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:27.917923    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"]
Jul 19 14:41:27.918557 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:27.918169    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn" podUID="58988304-b2d1-4f0b-80eb-7d8144edc73b" containerName="maas-api" containerID="cri-o://3195909aa6dd672db81ceebff11d24c11afb964b80cc55a97001b9ddef772261" gracePeriod=30
Jul 19 14:41:27.939389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:27.939363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:28.116571 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.116539    2574 generic.go:356] "Generic (PLEG): container finished" podID="58988304-b2d1-4f0b-80eb-7d8144edc73b" containerID="3195909aa6dd672db81ceebff11d24c11afb964b80cc55a97001b9ddef772261" exitCode=0
Jul 19 14:41:28.116723 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.116615    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn" event={"ID":"58988304-b2d1-4f0b-80eb-7d8144edc73b","Type":"ContainerDied","Data":"3195909aa6dd672db81ceebff11d24c11afb964b80cc55a97001b9ddef772261"}
Jul 19 14:41:28.163161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.163140    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:41:28.204451 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.204376    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs7nt\" (UniqueName: \"kubernetes.io/projected/58988304-b2d1-4f0b-80eb-7d8144edc73b-kube-api-access-vs7nt\") pod \"58988304-b2d1-4f0b-80eb-7d8144edc73b\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") "
Jul 19 14:41:28.204608 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.204479    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") pod \"58988304-b2d1-4f0b-80eb-7d8144edc73b\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") "
Jul 19 14:41:28.204608 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.204559    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58988304-b2d1-4f0b-80eb-7d8144edc73b-openshift-service-ca\") pod \"58988304-b2d1-4f0b-80eb-7d8144edc73b\" (UID: \"58988304-b2d1-4f0b-80eb-7d8144edc73b\") "
Jul 19 14:41:28.204930 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.204899    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58988304-b2d1-4f0b-80eb-7d8144edc73b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58988304-b2d1-4f0b-80eb-7d8144edc73b" (UID: "58988304-b2d1-4f0b-80eb-7d8144edc73b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:41:28.206431 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.206400    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58988304-b2d1-4f0b-80eb-7d8144edc73b" (UID: "58988304-b2d1-4f0b-80eb-7d8144edc73b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:41:28.206548 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.206449    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58988304-b2d1-4f0b-80eb-7d8144edc73b-kube-api-access-vs7nt" (OuterVolumeSpecName: "kube-api-access-vs7nt") pod "58988304-b2d1-4f0b-80eb-7d8144edc73b" (UID: "58988304-b2d1-4f0b-80eb-7d8144edc73b"). InnerVolumeSpecName "kube-api-access-vs7nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:41:28.305781 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.305756    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58988304-b2d1-4f0b-80eb-7d8144edc73b-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:41:28.305781 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.305778    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58988304-b2d1-4f0b-80eb-7d8144edc73b-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:41:28.305930 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:28.305788    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs7nt\" (UniqueName: \"kubernetes.io/projected/58988304-b2d1-4f0b-80eb-7d8144edc73b-kube-api-access-vs7nt\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:41:29.122344 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:29.122314    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"
Jul 19 14:41:29.122798 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:29.122366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn" event={"ID":"58988304-b2d1-4f0b-80eb-7d8144edc73b","Type":"ContainerDied","Data":"1272403541b83593c06ec36c7d10981920348f7a62dbf8a18fc4ba889f624fc1"}
Jul 19 14:41:29.122798 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:29.122430    2574 scope.go:117] "RemoveContainer" containerID="3195909aa6dd672db81ceebff11d24c11afb964b80cc55a97001b9ddef772261"
Jul 19 14:41:29.137072 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:29.137049    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"]
Jul 19 14:41:29.140310 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:29.140289    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35312369-54f947d48b-fdgrn"]
Jul 19 14:41:30.396599 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:30.396566    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58988304-b2d1-4f0b-80eb-7d8144edc73b" path="/var/lib/kubelet/pods/58988304-b2d1-4f0b-80eb-7d8144edc73b/volumes"
Jul 19 14:41:32.632341 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:32.632308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:32.836161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:32.836132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:33.030514 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:33.030419    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:33.510599 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:33.510563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:33.608782 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:33.608752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:33.660812 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:33.660775    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:35.257755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:35.257724    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.067220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:41.067178    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.168167 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:41.168116    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.268223 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:41.268191    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.320286 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:41.320203    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.750220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:41.750188    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:45.364746 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:45.364701    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:45.465255 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:41:45.465225    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:07.706877 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:07.706839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:13.842884 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:13.842844    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:13.950584 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:13.950549    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:14.046716 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:14.046670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:29.916123 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:29.916091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:35.561726 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:35.561688    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:35.618531 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:35.618500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:35.701321 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:35.701288    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:35.802694 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:35.802661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:46.301731 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:46.301697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:47.109453 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:47.109409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:53.187315 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:53.187277    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:53.288581 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:53.288542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:53.387854 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:53.387822    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:53.487931 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:53.487855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:54.260991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:54.260878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:54.361268 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:54.361213    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:54.460309 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:54.460280    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:56.074088 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:42:56.074058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:01.821365 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:01.821306    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:01.921415 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:01.921377    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:02.167953 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:02.167922    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:43:02.173695 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:02.173678    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:43:02.502142 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:02.502055    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:08.064780 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:08.064748    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:08.164611 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:08.164583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:08.300731 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:08.300701    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:08.733974 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:08.733924    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:08.887108 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:08.887073    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:12.001217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.001165    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:12.095902 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.095866    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"]
Jul 19 14:43:12.096435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.096413    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58988304-b2d1-4f0b-80eb-7d8144edc73b" containerName="maas-api"
Jul 19 14:43:12.096565 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.096438    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="58988304-b2d1-4f0b-80eb-7d8144edc73b" containerName="maas-api"
Jul 19 14:43:12.096631 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.096571    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="58988304-b2d1-4f0b-80eb-7d8144edc73b" containerName="maas-api"
Jul 19 14:43:12.097178 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.097158    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:12.099973 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.099953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.102221 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.102201    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b3de6a98-serving-cert\""
Jul 19 14:43:12.110164 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.110142    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"]
Jul 19 14:43:12.168166 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.168135    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:12.224896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.224870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhnf9\" (UniqueName: \"kubernetes.io/projected/31ffd769-044a-4f05-930d-83224894fad2-kube-api-access-jhnf9\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.224990 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.224905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31ffd769-044a-4f05-930d-83224894fad2-maas-api-tls\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.224990 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.224936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ffd769-044a-4f05-930d-83224894fad2-openshift-service-ca\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.268075 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.268013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:12.326217 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.326193    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31ffd769-044a-4f05-930d-83224894fad2-maas-api-tls\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.326308 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.326240    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ffd769-044a-4f05-930d-83224894fad2-openshift-service-ca\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.326347 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.326331    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhnf9\" (UniqueName: \"kubernetes.io/projected/31ffd769-044a-4f05-930d-83224894fad2-kube-api-access-jhnf9\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.326923 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.326904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ffd769-044a-4f05-930d-83224894fad2-openshift-service-ca\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.328711 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.328691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31ffd769-044a-4f05-930d-83224894fad2-maas-api-tls\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.334056 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.334033    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhnf9\" (UniqueName: \"kubernetes.io/projected/31ffd769-044a-4f05-930d-83224894fad2-kube-api-access-jhnf9\") pod \"maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.411475 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.411447    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:12.553842 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.553813    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"]
Jul 19 14:43:12.555647 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:43:12.555618    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ffd769_044a_4f05_930d_83224894fad2.slice/crio-f167678e7ec78fd621e4470f285764d1db4d71aa1eedd64dd01aa821496ced8b WatchSource:0}: Error finding container f167678e7ec78fd621e4470f285764d1db4d71aa1eedd64dd01aa821496ced8b: Status 404 returned error can't find the container with id f167678e7ec78fd621e4470f285764d1db4d71aa1eedd64dd01aa821496ced8b
Jul 19 14:43:12.588142 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.588110    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm" event={"ID":"31ffd769-044a-4f05-930d-83224894fad2","Type":"ContainerStarted","Data":"f167678e7ec78fd621e4470f285764d1db4d71aa1eedd64dd01aa821496ced8b"}
Jul 19 14:43:12.737614 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.737580    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:12.839286 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:12.839221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:13.594859 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:13.594813    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm" event={"ID":"31ffd769-044a-4f05-930d-83224894fad2","Type":"ContainerStarted","Data":"9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344"}
Jul 19 14:43:13.595235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:13.594932    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:13.610551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:13.610490    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm" podStartSLOduration=1.147963046 podStartE2EDuration="1.610476054s" totalImagesPullingTime="462.513008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:43:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:43:13.609375427 +0000 UTC m=+1811.912206460" watchObservedRunningTime="2026-07-19 14:43:13.610476054 +0000 UTC m=+1811.913307069"
Jul 19 14:43:16.504888 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:16.504850    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:17.161596 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:17.161562    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:17.258112 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:17.258085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:17.604110 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:17.604024    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:19.605389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:19.605360    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:23.307193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:23.307155    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:23.362749 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:23.362719    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:34.018733 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.018696    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"]
Jul 19 14:43:34.019205 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.018931    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm" podUID="31ffd769-044a-4f05-930d-83224894fad2" containerName="maas-api" containerID="cri-o://9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344" gracePeriod=30
Jul 19 14:43:34.050558 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.050512    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:34.265026 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.265004    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:34.418590 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.418564    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhnf9\" (UniqueName: \"kubernetes.io/projected/31ffd769-044a-4f05-930d-83224894fad2-kube-api-access-jhnf9\") pod \"31ffd769-044a-4f05-930d-83224894fad2\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") "
Jul 19 14:43:34.418746 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.418649    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31ffd769-044a-4f05-930d-83224894fad2-maas-api-tls\") pod \"31ffd769-044a-4f05-930d-83224894fad2\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") "
Jul 19 14:43:34.418746 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.418676    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ffd769-044a-4f05-930d-83224894fad2-openshift-service-ca\") pod \"31ffd769-044a-4f05-930d-83224894fad2\" (UID: \"31ffd769-044a-4f05-930d-83224894fad2\") "
Jul 19 14:43:34.419086 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.419048    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ffd769-044a-4f05-930d-83224894fad2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31ffd769-044a-4f05-930d-83224894fad2" (UID: "31ffd769-044a-4f05-930d-83224894fad2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:43:34.420569 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.420544    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31ffd769-044a-4f05-930d-83224894fad2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31ffd769-044a-4f05-930d-83224894fad2" (UID: "31ffd769-044a-4f05-930d-83224894fad2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:43:34.420678 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.420604    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ffd769-044a-4f05-930d-83224894fad2-kube-api-access-jhnf9" (OuterVolumeSpecName: "kube-api-access-jhnf9") pod "31ffd769-044a-4f05-930d-83224894fad2" (UID: "31ffd769-044a-4f05-930d-83224894fad2"). InnerVolumeSpecName "kube-api-access-jhnf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:43:34.520185 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.520158    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhnf9\" (UniqueName: \"kubernetes.io/projected/31ffd769-044a-4f05-930d-83224894fad2-kube-api-access-jhnf9\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:43:34.520185 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.520182    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31ffd769-044a-4f05-930d-83224894fad2-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:43:34.520336 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.520192    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ffd769-044a-4f05-930d-83224894fad2-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:43:34.700292 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.700217    2574 generic.go:356] "Generic (PLEG): container finished" podID="31ffd769-044a-4f05-930d-83224894fad2" containerID="9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344" exitCode=0
Jul 19 14:43:34.700292 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.700263    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"
Jul 19 14:43:34.700514 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.700312    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm" event={"ID":"31ffd769-044a-4f05-930d-83224894fad2","Type":"ContainerDied","Data":"9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344"}
Jul 19 14:43:34.700514 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.700372    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm" event={"ID":"31ffd769-044a-4f05-930d-83224894fad2","Type":"ContainerDied","Data":"f167678e7ec78fd621e4470f285764d1db4d71aa1eedd64dd01aa821496ced8b"}
Jul 19 14:43:34.700514 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.700401    2574 scope.go:117] "RemoveContainer" containerID="9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344"
Jul 19 14:43:34.709590 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.709574    2574 scope.go:117] "RemoveContainer" containerID="9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344"
Jul 19 14:43:34.709829 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:43:34.709809    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344\": container with ID starting with 9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344 not found: ID does not exist" containerID="9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344"
Jul 19 14:43:34.709896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.709836    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344"} err="failed to get container status \"9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344\": rpc error: code = NotFound desc = could not find container \"9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344\": container with ID starting with 9c195121d3af08dea0a569e88578dcd108bf7b053418ec000a784d1302643344 not found: ID does not exist"
Jul 19 14:43:34.721464 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.721442    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"]
Jul 19 14:43:34.723059 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:34.723041    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b3de6a98-5859c45c58-9jpnm"]
Jul 19 14:43:36.396923 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:36.396890    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ffd769-044a-4f05-930d-83224894fad2" path="/var/lib/kubelet/pods/31ffd769-044a-4f05-930d-83224894fad2/volumes"
Jul 19 14:43:38.729148 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:38.729111    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:39.104362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:39.104281    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:44.833669 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:44.833632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:44.886130 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:44.886102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:45.131605 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:45.131574    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:45.321291 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:45.321259    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:45.445269 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:45.445198    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:45.544116 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:45.544087    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:45.672880 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:45.672845    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:51.337287 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:51.337256    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:51.442144 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:51.442117    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:51.536071 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:51.536041    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:51.686005 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:51.685976    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:57.214471 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:57.214429    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:57.278455 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:43:57.278424    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:04.029797 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:04.029755    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:04.347215 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:04.347136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:04.446598 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:04.446561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:07.214691 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.214645    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:07.307402 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.307368    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:07.397005 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.396976    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"]
Jul 19 14:44:07.397396 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.397378    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ffd769-044a-4f05-930d-83224894fad2" containerName="maas-api"
Jul 19 14:44:07.397487 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.397397    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ffd769-044a-4f05-930d-83224894fad2" containerName="maas-api"
Jul 19 14:44:07.397556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.397491    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ffd769-044a-4f05-930d-83224894fad2" containerName="maas-api"
Jul 19 14:44:07.400885 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.400863    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.403235 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.403215    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e1c86cb1-serving-cert\""
Jul 19 14:44:07.410151 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.410133    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"]
Jul 19 14:44:07.428356 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.428322    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:07.475393 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.475332    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfd4l\" (UniqueName: \"kubernetes.io/projected/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-kube-api-access-hfd4l\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.475493 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.475390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-openshift-service-ca\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.475493 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.475477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-maas-api-tls\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.529501 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.529474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:07.576321 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.576292    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfd4l\" (UniqueName: \"kubernetes.io/projected/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-kube-api-access-hfd4l\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.576420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.576344    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-openshift-service-ca\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.576420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.576392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-maas-api-tls\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.576987 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.576956    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-openshift-service-ca\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.578880 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.578857    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-maas-api-tls\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.583926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.583904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfd4l\" (UniqueName: \"kubernetes.io/projected/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-kube-api-access-hfd4l\") pod \"maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.714890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.714865    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:07.840810 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.840757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:07.848140 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.848118    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"]
Jul 19 14:44:07.850475 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:44:07.850450    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e28b30_8d9f_4c3b_b540_5ed51a7874b2.slice/crio-d1ac23c86368e60221d3f067d7a94a546a72a1afb86fb484146c49989d837a94 WatchSource:0}: Error finding container d1ac23c86368e60221d3f067d7a94a546a72a1afb86fb484146c49989d837a94: Status 404 returned error can't find the container with id d1ac23c86368e60221d3f067d7a94a546a72a1afb86fb484146c49989d837a94
Jul 19 14:44:07.852080 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.852061    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:44:07.941150 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:07.941125    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:08.850348 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:08.850312    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2" event={"ID":"14e28b30-8d9f-4c3b-b540-5ed51a7874b2","Type":"ContainerStarted","Data":"f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29"}
Jul 19 14:44:08.850779 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:08.850379    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:08.850779 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:08.850403    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2" event={"ID":"14e28b30-8d9f-4c3b-b540-5ed51a7874b2","Type":"ContainerStarted","Data":"d1ac23c86368e60221d3f067d7a94a546a72a1afb86fb484146c49989d837a94"}
Jul 19 14:44:08.866610 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:08.866553    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2" podStartSLOduration=1.489805506 podStartE2EDuration="1.866537164s" totalImagesPullingTime="376.731658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:44:08.864603951 +0000 UTC m=+1867.167434967" watchObservedRunningTime="2026-07-19 14:44:08.866537164 +0000 UTC m=+1867.169368170"
Jul 19 14:44:11.688547 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:11.688491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:14.860691 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:14.860667    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:44:22.004036 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:22.004004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:27.755635 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:44:27.755546    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:00.384753 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.384722    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"]
Jul 19 14:45:00.385220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.384927    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2" podUID="14e28b30-8d9f-4c3b-b540-5ed51a7874b2" containerName="maas-api" containerID="cri-o://f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29" gracePeriod=30
Jul 19 14:45:00.412876 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.412841    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:00.623580 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.623554    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:45:00.745139 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.745041    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfd4l\" (UniqueName: \"kubernetes.io/projected/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-kube-api-access-hfd4l\") pod \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") "
Jul 19 14:45:00.745139 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.745090    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-openshift-service-ca\") pod \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") "
Jul 19 14:45:00.745139 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.745137    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-maas-api-tls\") pod \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\" (UID: \"14e28b30-8d9f-4c3b-b540-5ed51a7874b2\") "
Jul 19 14:45:00.745582 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.745554    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14e28b30-8d9f-4c3b-b540-5ed51a7874b2" (UID: "14e28b30-8d9f-4c3b-b540-5ed51a7874b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:45:00.747299 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.747263    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-kube-api-access-hfd4l" (OuterVolumeSpecName: "kube-api-access-hfd4l") pod "14e28b30-8d9f-4c3b-b540-5ed51a7874b2" (UID: "14e28b30-8d9f-4c3b-b540-5ed51a7874b2"). InnerVolumeSpecName "kube-api-access-hfd4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:45:00.747576 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.747547    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14e28b30-8d9f-4c3b-b540-5ed51a7874b2" (UID: "14e28b30-8d9f-4c3b-b540-5ed51a7874b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:45:00.846195 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.846155    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfd4l\" (UniqueName: \"kubernetes.io/projected/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-kube-api-access-hfd4l\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:45:00.846195 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.846188    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:45:00.846195 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:00.846199    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e28b30-8d9f-4c3b-b540-5ed51a7874b2-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:45:01.082220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.082133    2574 generic.go:356] "Generic (PLEG): container finished" podID="14e28b30-8d9f-4c3b-b540-5ed51a7874b2" containerID="f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29" exitCode=0
Jul 19 14:45:01.082220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.082180    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"
Jul 19 14:45:01.082220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.082189    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2" event={"ID":"14e28b30-8d9f-4c3b-b540-5ed51a7874b2","Type":"ContainerDied","Data":"f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29"}
Jul 19 14:45:01.082459 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.082240    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2" event={"ID":"14e28b30-8d9f-4c3b-b540-5ed51a7874b2","Type":"ContainerDied","Data":"d1ac23c86368e60221d3f067d7a94a546a72a1afb86fb484146c49989d837a94"}
Jul 19 14:45:01.082459 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.082269    2574 scope.go:117] "RemoveContainer" containerID="f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29"
Jul 19 14:45:01.091569 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.091547    2574 scope.go:117] "RemoveContainer" containerID="f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29"
Jul 19 14:45:01.091855 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:45:01.091832    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29\": container with ID starting with f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29 not found: ID does not exist" containerID="f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29"
Jul 19 14:45:01.091924 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.091866    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29"} err="failed to get container status \"f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29\": rpc error: code = NotFound desc = could not find container \"f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29\": container with ID starting with f3af97d86aded121cece661be253b9564da35f4ff1c7e5cc7900711adb7a5e29 not found: ID does not exist"
Jul 19 14:45:01.102755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.102730    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"]
Jul 19 14:45:01.106168 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:01.106147    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1c86cb1-5bc8d9d745-z8wl2"]
Jul 19 14:45:02.397256 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:02.397221    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e28b30-8d9f-4c3b-b540-5ed51a7874b2" path="/var/lib/kubelet/pods/14e28b30-8d9f-4c3b-b540-5ed51a7874b2/volumes"
Jul 19 14:45:05.051631 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:05.051598    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:07.287892 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:07.287851    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:12.312376 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:12.312338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:48.589834 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:48.589801    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:49.101790 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:49.101755    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:54.595362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:54.595273    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:54.652959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:54.652925    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:54.886139 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:54.886105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:55.123670 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:55.123636    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:56.175563 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:56.175512    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:56.275144 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:56.275108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:56.375297 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:56.375265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.109147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:57.109112    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.209093 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:57.209056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.308930 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:57.308904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:58.681435 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:45:58.681404    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:04.562577 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:04.562546    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:04.616232 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:04.616204    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:05.225181 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:05.225140    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:11.057447 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:11.057416    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:11.163262 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:11.163227    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:11.259709 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:11.259665    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:21.798029 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:21.797975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:22.146530 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:22.146479    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:28.671768 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:28.671703    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:43.117145 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:43.117105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:46.124111 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:46.124076    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:46.225364 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:46.225331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:49.441404 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:49.441364    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:49.510715 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:49.510673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:49.780179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:49.780087    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:50.088028 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:50.087948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:50.184389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:50.184357    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:52.992219 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:52.992173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:53.089808 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.089771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:53.217309 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.217278    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"]
Jul 19 14:46:53.217757 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.217737    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e28b30-8d9f-4c3b-b540-5ed51a7874b2" containerName="maas-api"
Jul 19 14:46:53.217891 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.217773    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e28b30-8d9f-4c3b-b540-5ed51a7874b2" containerName="maas-api"
Jul 19 14:46:53.217955 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.217893    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e28b30-8d9f-4c3b-b540-5ed51a7874b2" containerName="maas-api"
Jul 19 14:46:53.220962 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.220939    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.223243 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.223224    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-763d0d-serving-cert\""
Jul 19 14:46:53.231069 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.231039    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"]
Jul 19 14:46:53.250208 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.250128    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:53.357131 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.357096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqsrf\" (UniqueName: \"kubernetes.io/projected/5fbec726-cf40-4fba-8895-e7652b5278cb-kube-api-access-lqsrf\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.357320 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.357144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fbec726-cf40-4fba-8895-e7652b5278cb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.357320 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.357205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.458657 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.458621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqsrf\" (UniqueName: \"kubernetes.io/projected/5fbec726-cf40-4fba-8895-e7652b5278cb-kube-api-access-lqsrf\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.458847 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.458668    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fbec726-cf40-4fba-8895-e7652b5278cb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.458847 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.458702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.458847 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:46:53.458811    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-serving-cert: secret "maas-api-e2e-inf-a-763d0d-serving-cert" not found
Jul 19 14:46:53.459010 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:46:53.458869    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls podName:5fbec726-cf40-4fba-8895-e7652b5278cb nodeName:}" failed. No retries permitted until 2026-07-19 14:46:53.958853924 +0000 UTC m=+2032.261684919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls") pod "maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" (UID: "5fbec726-cf40-4fba-8895-e7652b5278cb") : secret "maas-api-e2e-inf-a-763d0d-serving-cert" not found
Jul 19 14:46:53.459331 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.459311    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fbec726-cf40-4fba-8895-e7652b5278cb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.466772 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.466751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqsrf\" (UniqueName: \"kubernetes.io/projected/5fbec726-cf40-4fba-8895-e7652b5278cb-kube-api-access-lqsrf\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.535971 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.535880    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:53.673446 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.673408    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:53.775405 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.775374    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:53.963890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.963855    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:53.966434 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:53.966372    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:54.133737 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:54.133705    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:54.265885 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:54.265861    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"]
Jul 19 14:46:54.269185 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:46:54.269152    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbec726_cf40_4fba_8895_e7652b5278cb.slice/crio-25f2a7ace7272031b2e86e5bca32b3f9b322e29e783fce8a191ecc400319a922 WatchSource:0}: Error finding container 25f2a7ace7272031b2e86e5bca32b3f9b322e29e783fce8a191ecc400319a922: Status 404 returned error can't find the container with id 25f2a7ace7272031b2e86e5bca32b3f9b322e29e783fce8a191ecc400319a922
Jul 19 14:46:54.594429 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:54.594346    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" event={"ID":"5fbec726-cf40-4fba-8895-e7652b5278cb","Type":"ContainerStarted","Data":"25f2a7ace7272031b2e86e5bca32b3f9b322e29e783fce8a191ecc400319a922"}
Jul 19 14:46:55.600296 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:55.600259    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" event={"ID":"5fbec726-cf40-4fba-8895-e7652b5278cb","Type":"ContainerStarted","Data":"b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb"}
Jul 19 14:46:55.600693 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:55.600402    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:46:55.615540 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:55.615461    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" podStartSLOduration=2.221966401 podStartE2EDuration="2.615445807s" totalImagesPullingTime="393.479406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:55.614794625 +0000 UTC m=+2033.917625641" watchObservedRunningTime="2026-07-19 14:46:55.615445807 +0000 UTC m=+2033.918276867"
Jul 19 14:46:57.511960 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:57.511922    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:57.846940 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:57.846854    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:58.156458 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:58.156414    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:58.890172 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:46:58.890144    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.015660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.015621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.113408 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.113362    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.202388 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.202355    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"]
Jul 19 14:47:01.206441 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.206417    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.208855 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.208785    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a96ad8-serving-cert\""
Jul 19 14:47:01.215750 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.215726    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"]
Jul 19 14:47:01.223910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.223885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/117da31b-db95-449e-8eae-04b770f0fb86-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.224047 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.223928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfls4\" (UniqueName: \"kubernetes.io/projected/117da31b-db95-449e-8eae-04b770f0fb86-kube-api-access-mfls4\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.224111 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.224047    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.235179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.235150    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.325257 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.325174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/117da31b-db95-449e-8eae-04b770f0fb86-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.325257 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.325219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfls4\" (UniqueName: \"kubernetes.io/projected/117da31b-db95-449e-8eae-04b770f0fb86-kube-api-access-mfls4\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.325257 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.325251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.325490 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:47:01.325376    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-serving-cert: secret "maas-api-e2e-inf-b-a96ad8-serving-cert" not found
Jul 19 14:47:01.325490 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:47:01.325432    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls podName:117da31b-db95-449e-8eae-04b770f0fb86 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:01.82541532 +0000 UTC m=+2040.128246314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls") pod "maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" (UID: "117da31b-db95-449e-8eae-04b770f0fb86") : secret "maas-api-e2e-inf-b-a96ad8-serving-cert" not found
Jul 19 14:47:01.325825 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.325805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/117da31b-db95-449e-8eae-04b770f0fb86-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.334778 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.334753    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfls4\" (UniqueName: \"kubernetes.io/projected/117da31b-db95-449e-8eae-04b770f0fb86-kube-api-access-mfls4\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.583538 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.583434    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.609946 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.609920    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:47:01.702774 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.702739    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.803389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.803356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:01.828601 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.828565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:01.831025 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:01.831002    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls\") pod \"maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:02.121697 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:02.121663    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:02.262210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:02.262184    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"]
Jul 19 14:47:02.264490 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:47:02.264460    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117da31b_db95_449e_8eae_04b770f0fb86.slice/crio-14a2f56647da499a7b5f812f36b0fa102eb9fefc6d7b1b4fd0c3feb5d6f32382 WatchSource:0}: Error finding container 14a2f56647da499a7b5f812f36b0fa102eb9fefc6d7b1b4fd0c3feb5d6f32382: Status 404 returned error can't find the container with id 14a2f56647da499a7b5f812f36b0fa102eb9fefc6d7b1b4fd0c3feb5d6f32382
Jul 19 14:47:02.632817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:02.632781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" event={"ID":"117da31b-db95-449e-8eae-04b770f0fb86","Type":"ContainerStarted","Data":"18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e"}
Jul 19 14:47:02.632961 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:02.632833    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" event={"ID":"117da31b-db95-449e-8eae-04b770f0fb86","Type":"ContainerStarted","Data":"14a2f56647da499a7b5f812f36b0fa102eb9fefc6d7b1b4fd0c3feb5d6f32382"}
Jul 19 14:47:02.632961 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:02.632898    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:02.663434 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:02.663385    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" podStartSLOduration=1.37282668 podStartE2EDuration="1.663368265s" totalImagesPullingTime="290.541585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:02.662266483 +0000 UTC m=+2040.965097499" watchObservedRunningTime="2026-07-19 14:47:02.663368265 +0000 UTC m=+2040.966199280"
Jul 19 14:47:05.541797 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:05.541759    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:08.643051 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:08.643023    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:47:11.299376 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:11.299340    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.395856 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:14.395814    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.497764 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:14.497726    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:22.414985 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:22.414945    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:22.516237 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:22.516203    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:31.025308 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:31.025221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:36.829134 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:36.829099    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:48.152399 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:48.152364    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:48.251752 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:48.251717    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:53.901264 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:53.901231    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:54.752660 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:54.752631    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:54.893319 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:54.893288    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"]
Jul 19 14:47:54.896958 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:54.896932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:47:54.899173 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:54.899153    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79vn9\""
Jul 19 14:47:54.904968 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:54.904470    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"]
Jul 19 14:47:55.010091 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.010003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94b62c0a-a0f3-4ae1-9136-06737b646915-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-763d0d-x5629\" (UID: \"94b62c0a-a0f3-4ae1-9136-06737b646915\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:47:55.110724 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.110693    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94b62c0a-a0f3-4ae1-9136-06737b646915-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-763d0d-x5629\" (UID: \"94b62c0a-a0f3-4ae1-9136-06737b646915\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:47:55.111291 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.111271    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94b62c0a-a0f3-4ae1-9136-06737b646915-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-763d0d-x5629\" (UID: \"94b62c0a-a0f3-4ae1-9136-06737b646915\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:47:55.209716 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.209682    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:47:55.338464 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.338438    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"]
Jul 19 14:47:55.340734 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:47:55.340708    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b62c0a_a0f3_4ae1_9136_06737b646915.slice/crio-dbe99e0227ce602406785c79b80191fa2c98c66170add49a313c723d9d80c091 WatchSource:0}: Error finding container dbe99e0227ce602406785c79b80191fa2c98c66170add49a313c723d9d80c091: Status 404 returned error can't find the container with id dbe99e0227ce602406785c79b80191fa2c98c66170add49a313c723d9d80c091
Jul 19 14:47:55.394634 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.394591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:55.495778 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.495737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:55.885479 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:55.885444    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629" event={"ID":"94b62c0a-a0f3-4ae1-9136-06737b646915","Type":"ContainerStarted","Data":"dbe99e0227ce602406785c79b80191fa2c98c66170add49a313c723d9d80c091"}
Jul 19 14:47:57.897397 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:57.897365    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629" event={"ID":"94b62c0a-a0f3-4ae1-9136-06737b646915","Type":"ContainerStarted","Data":"8d5c45d14faa7f475f51208670cf87d2eb63d4b7efbefbf5409c6d88cecec890"}
Jul 19 14:47:57.912237 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:57.912190    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629" podStartSLOduration=1.7821725910000001 podStartE2EDuration="3.912172876s" totalImagesPullingTime="2.130000285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:57.910689143 +0000 UTC m=+2096.213520165" watchObservedRunningTime="2026-07-19 14:47:57.912172876 +0000 UTC m=+2096.215003891"
Jul 19 14:47:58.903377 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:58.903340    2574 generic.go:356] "Generic (PLEG): container finished" podID="94b62c0a-a0f3-4ae1-9136-06737b646915" containerID="8d5c45d14faa7f475f51208670cf87d2eb63d4b7efbefbf5409c6d88cecec890" exitCode=0
Jul 19 14:47:58.903810 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:47:58.903422    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629" event={"ID":"94b62c0a-a0f3-4ae1-9136-06737b646915","Type":"ContainerDied","Data":"8d5c45d14faa7f475f51208670cf87d2eb63d4b7efbefbf5409c6d88cecec890"}
Jul 19 14:48:00.033669 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.033642    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:48:00.162544 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.162448    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94b62c0a-a0f3-4ae1-9136-06737b646915-maas-api-service-ca\") pod \"94b62c0a-a0f3-4ae1-9136-06737b646915\" (UID: \"94b62c0a-a0f3-4ae1-9136-06737b646915\") "
Jul 19 14:48:00.162785 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.162764    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94b62c0a-a0f3-4ae1-9136-06737b646915-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "94b62c0a-a0f3-4ae1-9136-06737b646915" (UID: "94b62c0a-a0f3-4ae1-9136-06737b646915"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:48:00.263371 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.263340    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94b62c0a-a0f3-4ae1-9136-06737b646915-maas-api-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:00.914371 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.914340    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"
Jul 19 14:48:00.914554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.914376    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629" event={"ID":"94b62c0a-a0f3-4ae1-9136-06737b646915","Type":"ContainerDied","Data":"dbe99e0227ce602406785c79b80191fa2c98c66170add49a313c723d9d80c091"}
Jul 19 14:48:00.914554 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:00.914420    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbe99e0227ce602406785c79b80191fa2c98c66170add49a313c723d9d80c091"
Jul 19 14:48:02.195991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:02.195960    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:48:02.202245 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:02.202226    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:48:05.255664 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.255636    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"]
Jul 19 14:48:05.256078 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.255880    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" podUID="5fbec726-cf40-4fba-8895-e7652b5278cb" containerName="maas-api" containerID="cri-o://b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb" gracePeriod=30
Jul 19 14:48:05.281055 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.281024    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:05.697668 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.697645    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:48:05.811697 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.811614    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls\") pod \"5fbec726-cf40-4fba-8895-e7652b5278cb\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") "
Jul 19 14:48:05.811859 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.811728    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqsrf\" (UniqueName: \"kubernetes.io/projected/5fbec726-cf40-4fba-8895-e7652b5278cb-kube-api-access-lqsrf\") pod \"5fbec726-cf40-4fba-8895-e7652b5278cb\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") "
Jul 19 14:48:05.811859 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.811758    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fbec726-cf40-4fba-8895-e7652b5278cb-openshift-service-ca\") pod \"5fbec726-cf40-4fba-8895-e7652b5278cb\" (UID: \"5fbec726-cf40-4fba-8895-e7652b5278cb\") "
Jul 19 14:48:05.812163 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.812138    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fbec726-cf40-4fba-8895-e7652b5278cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fbec726-cf40-4fba-8895-e7652b5278cb" (UID: "5fbec726-cf40-4fba-8895-e7652b5278cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:48:05.813693 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.813671    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fbec726-cf40-4fba-8895-e7652b5278cb" (UID: "5fbec726-cf40-4fba-8895-e7652b5278cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:48:05.813777 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.813724    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fbec726-cf40-4fba-8895-e7652b5278cb-kube-api-access-lqsrf" (OuterVolumeSpecName: "kube-api-access-lqsrf") pod "5fbec726-cf40-4fba-8895-e7652b5278cb" (UID: "5fbec726-cf40-4fba-8895-e7652b5278cb"). InnerVolumeSpecName "kube-api-access-lqsrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:48:05.913085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.913053    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqsrf\" (UniqueName: \"kubernetes.io/projected/5fbec726-cf40-4fba-8895-e7652b5278cb-kube-api-access-lqsrf\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:05.913085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.913079    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fbec726-cf40-4fba-8895-e7652b5278cb-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:05.913085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.913088    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fbec726-cf40-4fba-8895-e7652b5278cb-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:05.938688 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.938660    2574 generic.go:356] "Generic (PLEG): container finished" podID="5fbec726-cf40-4fba-8895-e7652b5278cb" containerID="b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb" exitCode=0
Jul 19 14:48:05.938814 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.938709    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"
Jul 19 14:48:05.938814 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.938750    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" event={"ID":"5fbec726-cf40-4fba-8895-e7652b5278cb","Type":"ContainerDied","Data":"b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb"}
Jul 19 14:48:05.938814 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.938802    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625" event={"ID":"5fbec726-cf40-4fba-8895-e7652b5278cb","Type":"ContainerDied","Data":"25f2a7ace7272031b2e86e5bca32b3f9b322e29e783fce8a191ecc400319a922"}
Jul 19 14:48:05.938910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.938826    2574 scope.go:117] "RemoveContainer" containerID="b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb"
Jul 19 14:48:05.947951 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.947934    2574 scope.go:117] "RemoveContainer" containerID="b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb"
Jul 19 14:48:05.948191 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:48:05.948174    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb\": container with ID starting with b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb not found: ID does not exist" containerID="b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb"
Jul 19 14:48:05.948243 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.948199    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb"} err="failed to get container status \"b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb\": rpc error: code = NotFound desc = could not find container \"b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb\": container with ID starting with b514b4b676a76e33f1b6e3eaef36df88183a093f57f31ef7f8175b42849898fb not found: ID does not exist"
Jul 19 14:48:05.958959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.958936    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"]
Jul 19 14:48:05.964051 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:05.964034    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-763d0d-76f77fb9f6-cj625"]
Jul 19 14:48:06.397309 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:06.397277    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fbec726-cf40-4fba-8895-e7652b5278cb" path="/var/lib/kubelet/pods/5fbec726-cf40-4fba-8895-e7652b5278cb/volumes"
Jul 19 14:48:09.959834 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:09.959800    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"]
Jul 19 14:48:09.964195 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:09.964169    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-763d0d-x5629"]
Jul 19 14:48:09.965164 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:09.965139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:10.327647 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:10.327553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:10.396682 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:10.396654    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94b62c0a-a0f3-4ae1-9136-06737b646915" path="/var/lib/kubelet/pods/94b62c0a-a0f3-4ae1-9136-06737b646915/volumes"
Jul 19 14:48:16.095475 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:16.095440    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:16.195478 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:16.195448    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:16.384558 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:16.384491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:16.619861 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:16.619832    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:17.309501 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:17.309447    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:17.409999 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:17.409968    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:27.114864 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.114829    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"]
Jul 19 14:48:27.115472 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.115085    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" podUID="117da31b-db95-449e-8eae-04b770f0fb86" containerName="maas-api" containerID="cri-o://18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e" gracePeriod=30
Jul 19 14:48:27.136483 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.136454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:27.364578 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.364554    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:48:27.499372 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.499283    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfls4\" (UniqueName: \"kubernetes.io/projected/117da31b-db95-449e-8eae-04b770f0fb86-kube-api-access-mfls4\") pod \"117da31b-db95-449e-8eae-04b770f0fb86\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") "
Jul 19 14:48:27.499372 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.499347    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/117da31b-db95-449e-8eae-04b770f0fb86-openshift-service-ca\") pod \"117da31b-db95-449e-8eae-04b770f0fb86\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") "
Jul 19 14:48:27.499372 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.499372    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls\") pod \"117da31b-db95-449e-8eae-04b770f0fb86\" (UID: \"117da31b-db95-449e-8eae-04b770f0fb86\") "
Jul 19 14:48:27.499748 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.499722    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117da31b-db95-449e-8eae-04b770f0fb86-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "117da31b-db95-449e-8eae-04b770f0fb86" (UID: "117da31b-db95-449e-8eae-04b770f0fb86"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:48:27.501388 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.501356    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "117da31b-db95-449e-8eae-04b770f0fb86" (UID: "117da31b-db95-449e-8eae-04b770f0fb86"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:48:27.501388 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.501378    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117da31b-db95-449e-8eae-04b770f0fb86-kube-api-access-mfls4" (OuterVolumeSpecName: "kube-api-access-mfls4") pod "117da31b-db95-449e-8eae-04b770f0fb86" (UID: "117da31b-db95-449e-8eae-04b770f0fb86"). InnerVolumeSpecName "kube-api-access-mfls4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:48:27.600941 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.600904    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/117da31b-db95-449e-8eae-04b770f0fb86-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:27.600941 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.600936    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfls4\" (UniqueName: \"kubernetes.io/projected/117da31b-db95-449e-8eae-04b770f0fb86-kube-api-access-mfls4\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:27.600941 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:27.600946    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/117da31b-db95-449e-8eae-04b770f0fb86-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:48:28.040703 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.040666    2574 generic.go:356] "Generic (PLEG): container finished" podID="117da31b-db95-449e-8eae-04b770f0fb86" containerID="18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e" exitCode=0
Jul 19 14:48:28.040890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.040709    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"
Jul 19 14:48:28.040890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.040754    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" event={"ID":"117da31b-db95-449e-8eae-04b770f0fb86","Type":"ContainerDied","Data":"18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e"}
Jul 19 14:48:28.040890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.040814    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh" event={"ID":"117da31b-db95-449e-8eae-04b770f0fb86","Type":"ContainerDied","Data":"14a2f56647da499a7b5f812f36b0fa102eb9fefc6d7b1b4fd0c3feb5d6f32382"}
Jul 19 14:48:28.040890 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.040845    2574 scope.go:117] "RemoveContainer" containerID="18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e"
Jul 19 14:48:28.049322 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.049306    2574 scope.go:117] "RemoveContainer" containerID="18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e"
Jul 19 14:48:28.049563 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:48:28.049540    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e\": container with ID starting with 18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e not found: ID does not exist" containerID="18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e"
Jul 19 14:48:28.049625 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.049574    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e"} err="failed to get container status \"18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e\": rpc error: code = NotFound desc = could not find container \"18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e\": container with ID starting with 18d24f59a730a670d734c5c2275564597b947351d0b4ef67743a7a2d1186e32e not found: ID does not exist"
Jul 19 14:48:28.060813 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.060791    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"]
Jul 19 14:48:28.067052 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.067032    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a96ad8-b8bcb4b54-wxwgh"]
Jul 19 14:48:28.400549 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:28.397705    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="117da31b-db95-449e-8eae-04b770f0fb86" path="/var/lib/kubelet/pods/117da31b-db95-449e-8eae-04b770f0fb86/volumes"
Jul 19 14:48:31.809005 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:31.808976    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:32.190043 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:32.190004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:37.776732 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:37.776693    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:38.849213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:38.849176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:38.973321 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:38.973285    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:39.505193 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:39.505162    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:41.489939 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.489910    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:41.589214 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.589183    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:41.711271 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711243    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"]
Jul 19 14:48:41.711641 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711628    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94b62c0a-a0f3-4ae1-9136-06737b646915" containerName="revoke-keys"
Jul 19 14:48:41.711705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711649    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="94b62c0a-a0f3-4ae1-9136-06737b646915" containerName="revoke-keys"
Jul 19 14:48:41.711705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711659    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fbec726-cf40-4fba-8895-e7652b5278cb" containerName="maas-api"
Jul 19 14:48:41.711705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711664    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fbec726-cf40-4fba-8895-e7652b5278cb" containerName="maas-api"
Jul 19 14:48:41.711705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711680    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="117da31b-db95-449e-8eae-04b770f0fb86" containerName="maas-api"
Jul 19 14:48:41.711705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711685    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="117da31b-db95-449e-8eae-04b770f0fb86" containerName="maas-api"
Jul 19 14:48:41.711869 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711762    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fbec726-cf40-4fba-8895-e7652b5278cb" containerName="maas-api"
Jul 19 14:48:41.711869 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711773    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="117da31b-db95-449e-8eae-04b770f0fb86" containerName="maas-api"
Jul 19 14:48:41.711869 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.711781    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="94b62c0a-a0f3-4ae1-9136-06737b646915" containerName="revoke-keys"
Jul 19 14:48:41.714772 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.714757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.717588 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.717561    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8e7a06-serving-cert\""
Jul 19 14:48:41.729925 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.729890    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"]
Jul 19 14:48:41.744081 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.744011    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:41.821842 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.821812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jbz\" (UniqueName: \"kubernetes.io/projected/6700eb65-6167-410d-8493-bbda36d7b573-kube-api-access-42jbz\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.822017 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.821866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.822017 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.821898    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6700eb65-6167-410d-8493-bbda36d7b573-openshift-service-ca\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.922589 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.922553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jbz\" (UniqueName: \"kubernetes.io/projected/6700eb65-6167-410d-8493-bbda36d7b573-kube-api-access-42jbz\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.922762 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.922609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.922762 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.922636    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6700eb65-6167-410d-8493-bbda36d7b573-openshift-service-ca\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.922882 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:48:41.922758    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-serving-cert: secret "maas-api-e2e-api-a-8e7a06-serving-cert" not found
Jul 19 14:48:41.922882 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:48:41.922840    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls podName:6700eb65-6167-410d-8493-bbda36d7b573 nodeName:}" failed. No retries permitted until 2026-07-19 14:48:42.422819942 +0000 UTC m=+2140.725650948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls") pod "maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" (UID: "6700eb65-6167-410d-8493-bbda36d7b573") : secret "maas-api-e2e-api-a-8e7a06-serving-cert" not found
Jul 19 14:48:41.923286 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.923264    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6700eb65-6167-410d-8493-bbda36d7b573-openshift-service-ca\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:41.931164 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:41.931142    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jbz\" (UniqueName: \"kubernetes.io/projected/6700eb65-6167-410d-8493-bbda36d7b573-kube-api-access-42jbz\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:42.079040 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.078950    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:42.273371 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.273337    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:42.377314 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.377282    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:42.426215 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.426182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:42.428852 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.428826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls\") pod \"maas-api-e2e-api-a-8e7a06-76df59967d-fggb5\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:42.624820 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.624790    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:42.751968 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:42.751941    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"]
Jul 19 14:48:42.753897 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:48:42.753869    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6700eb65_6167_410d_8493_bbda36d7b573.slice/crio-6f228bae7be26351122e9649bb76b72b01dd9b136ebff321e22704bac4141767 WatchSource:0}: Error finding container 6f228bae7be26351122e9649bb76b72b01dd9b136ebff321e22704bac4141767: Status 404 returned error can't find the container with id 6f228bae7be26351122e9649bb76b72b01dd9b136ebff321e22704bac4141767
Jul 19 14:48:43.111516 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:43.111485    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" event={"ID":"6700eb65-6167-410d-8493-bbda36d7b573","Type":"ContainerStarted","Data":"6f228bae7be26351122e9649bb76b72b01dd9b136ebff321e22704bac4141767"}
Jul 19 14:48:44.117733 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:44.117696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" event={"ID":"6700eb65-6167-410d-8493-bbda36d7b573","Type":"ContainerStarted","Data":"6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274"}
Jul 19 14:48:44.118114 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:44.117755    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:44.133294 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:44.133230    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" podStartSLOduration=2.743967929 podStartE2EDuration="3.13320457s" totalImagesPullingTime="389.236641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:44.132875637 +0000 UTC m=+2142.435706652" watchObservedRunningTime="2026-07-19 14:48:44.13320457 +0000 UTC m=+2142.436035587"
Jul 19 14:48:46.010700 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:46.010662    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:46.502089 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:46.502057    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:46.826364 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:46.826285    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:47.151166 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:47.151113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:49.623907 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.623867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:49.697556 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.697483    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:49.699179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.699158    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"]
Jul 19 14:48:49.703266 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.703246    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.705682 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.705665    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4b4917-serving-cert\""
Jul 19 14:48:49.713020 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.712978    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"]
Jul 19 14:48:49.760571 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.760540    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:49.791678 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.791652    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.791825 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.791772    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.791825 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.791797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwdrr\" (UniqueName: \"kubernetes.io/projected/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-kube-api-access-nwdrr\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.893004 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.892925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwdrr\" (UniqueName: \"kubernetes.io/projected/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-kube-api-access-nwdrr\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.893004 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.892965    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.893227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.893069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.893227 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:48:49.893155    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-serving-cert: secret "maas-api-e2e-api-b-4b4917-serving-cert" not found
Jul 19 14:48:49.893294 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:48:49.893250    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls podName:b6e1dc97-85be-4b6f-ae39-015e7267d2ac nodeName:}" failed. No retries permitted until 2026-07-19 14:48:50.393228085 +0000 UTC m=+2148.696059085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls") pod "maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" (UID: "b6e1dc97-85be-4b6f-ae39-015e7267d2ac") : secret "maas-api-e2e-api-b-4b4917-serving-cert" not found
Jul 19 14:48:49.893666 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.893651    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:49.907369 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:49.907343    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwdrr\" (UniqueName: \"kubernetes.io/projected/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-kube-api-access-nwdrr\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:50.074458 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.074422    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:50.128244 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.128212    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:48:50.271079 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.270996    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:50.371731 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.371697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:50.399035 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.399003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:50.401434 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.401410    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls\") pod \"maas-api-e2e-api-b-4b4917-79457d48c7-swcbv\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:50.616147 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.616113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:50.755552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:50.755450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"]
Jul 19 14:48:50.757884 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:48:50.757855    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e1dc97_85be_4b6f_ae39_015e7267d2ac.slice/crio-383a461caf4c2b09147a193af18df156e0fc7be00193f204b578dd477ef04ee3 WatchSource:0}: Error finding container 383a461caf4c2b09147a193af18df156e0fc7be00193f204b578dd477ef04ee3: Status 404 returned error can't find the container with id 383a461caf4c2b09147a193af18df156e0fc7be00193f204b578dd477ef04ee3
Jul 19 14:48:51.153221 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:51.153188    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" event={"ID":"b6e1dc97-85be-4b6f-ae39-015e7267d2ac","Type":"ContainerStarted","Data":"926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96"}
Jul 19 14:48:51.153328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:51.153244    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" event={"ID":"b6e1dc97-85be-4b6f-ae39-015e7267d2ac","Type":"ContainerStarted","Data":"383a461caf4c2b09147a193af18df156e0fc7be00193f204b578dd477ef04ee3"}
Jul 19 14:48:51.153389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:51.153374    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:48:51.169014 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:51.168967    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" podStartSLOduration=1.843186504 podStartE2EDuration="2.168952717s" totalImagesPullingTime="325.766213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:51.168390338 +0000 UTC m=+2149.471221354" watchObservedRunningTime="2026-07-19 14:48:51.168952717 +0000 UTC m=+2149.471783733"
Jul 19 14:48:54.051806 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:54.051768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:57.163120 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:48:57.163092    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:49:02.594325 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:02.594242    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:02.694448 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:02.694414    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:10.978080 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:10.978047    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:11.078995 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:11.078960    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:12.621995 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:12.621965    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:12.723480 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:12.723425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:17.848804 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:17.848766    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"]
Jul 19 14:49:17.849574 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:17.849539    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" podUID="6700eb65-6167-410d-8493-bbda36d7b573" containerName="maas-api" containerID="cri-o://6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274" gracePeriod=30
Jul 19 14:49:17.869605 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:17.869574    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:18.091824 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.091800    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:49:18.251480 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.251445    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls\") pod \"6700eb65-6167-410d-8493-bbda36d7b573\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") "
Jul 19 14:49:18.251705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.251548    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6700eb65-6167-410d-8493-bbda36d7b573-openshift-service-ca\") pod \"6700eb65-6167-410d-8493-bbda36d7b573\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") "
Jul 19 14:49:18.251705 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.251574    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42jbz\" (UniqueName: \"kubernetes.io/projected/6700eb65-6167-410d-8493-bbda36d7b573-kube-api-access-42jbz\") pod \"6700eb65-6167-410d-8493-bbda36d7b573\" (UID: \"6700eb65-6167-410d-8493-bbda36d7b573\") "
Jul 19 14:49:18.251956 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.251931    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6700eb65-6167-410d-8493-bbda36d7b573-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6700eb65-6167-410d-8493-bbda36d7b573" (UID: "6700eb65-6167-410d-8493-bbda36d7b573"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:49:18.253614 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.253586    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6700eb65-6167-410d-8493-bbda36d7b573" (UID: "6700eb65-6167-410d-8493-bbda36d7b573"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:49:18.253716 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.253629    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6700eb65-6167-410d-8493-bbda36d7b573-kube-api-access-42jbz" (OuterVolumeSpecName: "kube-api-access-42jbz") pod "6700eb65-6167-410d-8493-bbda36d7b573" (UID: "6700eb65-6167-410d-8493-bbda36d7b573"). InnerVolumeSpecName "kube-api-access-42jbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:49:18.281782 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.281755    2574 generic.go:356] "Generic (PLEG): container finished" podID="6700eb65-6167-410d-8493-bbda36d7b573" containerID="6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274" exitCode=0
Jul 19 14:49:18.281910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.281807    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"
Jul 19 14:49:18.281910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.281814    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" event={"ID":"6700eb65-6167-410d-8493-bbda36d7b573","Type":"ContainerDied","Data":"6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274"}
Jul 19 14:49:18.281910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.281854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5" event={"ID":"6700eb65-6167-410d-8493-bbda36d7b573","Type":"ContainerDied","Data":"6f228bae7be26351122e9649bb76b72b01dd9b136ebff321e22704bac4141767"}
Jul 19 14:49:18.281910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.281877    2574 scope.go:117] "RemoveContainer" containerID="6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274"
Jul 19 14:49:18.292458 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.292442    2574 scope.go:117] "RemoveContainer" containerID="6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274"
Jul 19 14:49:18.292755 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:49:18.292736    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274\": container with ID starting with 6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274 not found: ID does not exist" containerID="6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274"
Jul 19 14:49:18.292813 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.292763    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274"} err="failed to get container status \"6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274\": rpc error: code = NotFound desc = could not find container \"6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274\": container with ID starting with 6c73bd99bfa3abc35704b03458ac4b7607e035df8d0eee3f05aa7c1c333e9274 not found: ID does not exist"
Jul 19 14:49:18.304734 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.304706    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"]
Jul 19 14:49:18.306451 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.306432    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e7a06-76df59967d-fggb5"]
Jul 19 14:49:18.352794 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.352766    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6700eb65-6167-410d-8493-bbda36d7b573-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:49:18.352794 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.352791    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6700eb65-6167-410d-8493-bbda36d7b573-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:49:18.352935 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.352801    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42jbz\" (UniqueName: \"kubernetes.io/projected/6700eb65-6167-410d-8493-bbda36d7b573-kube-api-access-42jbz\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:49:18.396896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:18.396871    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6700eb65-6167-410d-8493-bbda36d7b573" path="/var/lib/kubelet/pods/6700eb65-6167-410d-8493-bbda36d7b573/volumes"
Jul 19 14:49:22.204388 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:22.204357    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:22.585081 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:22.584991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:28.095213 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:28.095174    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:28.196101 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:28.196063    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:28.367943 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:28.367895    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:28.588387 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:28.588356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:29.329432 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:29.329396    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:29.430021 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:29.429989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:43.980856 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:43.980819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:44.029018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.028981    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:44.081330 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.081296    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"]
Jul 19 14:49:44.081795 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.081759    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" podUID="b6e1dc97-85be-4b6f-ae39-015e7267d2ac" containerName="maas-api" containerID="cri-o://926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96" gracePeriod=30
Jul 19 14:49:44.104020 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.103988    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:44.328489 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.328466    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:49:44.385016 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.384988    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls\") pod \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") "
Jul 19 14:49:44.385185 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.385035    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-openshift-service-ca\") pod \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") "
Jul 19 14:49:44.385185 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.385071    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwdrr\" (UniqueName: \"kubernetes.io/projected/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-kube-api-access-nwdrr\") pod \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\" (UID: \"b6e1dc97-85be-4b6f-ae39-015e7267d2ac\") "
Jul 19 14:49:44.385440 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.385413    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6e1dc97-85be-4b6f-ae39-015e7267d2ac" (UID: "b6e1dc97-85be-4b6f-ae39-015e7267d2ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:49:44.387132 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.387109    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6e1dc97-85be-4b6f-ae39-015e7267d2ac" (UID: "b6e1dc97-85be-4b6f-ae39-015e7267d2ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:49:44.387249 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.387229    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-kube-api-access-nwdrr" (OuterVolumeSpecName: "kube-api-access-nwdrr") pod "b6e1dc97-85be-4b6f-ae39-015e7267d2ac" (UID: "b6e1dc97-85be-4b6f-ae39-015e7267d2ac"). InnerVolumeSpecName "kube-api-access-nwdrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:49:44.403729 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.403693    2574 generic.go:356] "Generic (PLEG): container finished" podID="b6e1dc97-85be-4b6f-ae39-015e7267d2ac" containerID="926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96" exitCode=0
Jul 19 14:49:44.403881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.403745    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"
Jul 19 14:49:44.403881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.403767    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" event={"ID":"b6e1dc97-85be-4b6f-ae39-015e7267d2ac","Type":"ContainerDied","Data":"926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96"}
Jul 19 14:49:44.403881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.403810    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv" event={"ID":"b6e1dc97-85be-4b6f-ae39-015e7267d2ac","Type":"ContainerDied","Data":"383a461caf4c2b09147a193af18df156e0fc7be00193f204b578dd477ef04ee3"}
Jul 19 14:49:44.403881 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.403824    2574 scope.go:117] "RemoveContainer" containerID="926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96"
Jul 19 14:49:44.412459 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.412438    2574 scope.go:117] "RemoveContainer" containerID="926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96"
Jul 19 14:49:44.412735 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:49:44.412717    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96\": container with ID starting with 926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96 not found: ID does not exist" containerID="926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96"
Jul 19 14:49:44.412790 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.412743    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96"} err="failed to get container status \"926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96\": rpc error: code = NotFound desc = could not find container \"926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96\": container with ID starting with 926dca2c9fe0a6b4b56b06339f59ec4813db99f96ae3ef5406d204fd94e40d96 not found: ID does not exist"
Jul 19 14:49:44.419451 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.419430    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"]
Jul 19 14:49:44.422339 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.422318    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4b4917-79457d48c7-swcbv"]
Jul 19 14:49:44.486161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.486130    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:49:44.486161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.486158    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:49:44.486356 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:44.486170    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwdrr\" (UniqueName: \"kubernetes.io/projected/b6e1dc97-85be-4b6f-ae39-015e7267d2ac-kube-api-access-nwdrr\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:49:46.396672 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:46.396638    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e1dc97-85be-4b6f-ae39-015e7267d2ac" path="/var/lib/kubelet/pods/b6e1dc97-85be-4b6f-ae39-015e7267d2ac/volumes"
Jul 19 14:49:48.811382 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:48.811344    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:49.183598 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:49.183564    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:55.063635 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:55.063602    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:55.167587 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:55.167551    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:55.333870 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:55.333787    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:55.563661 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:55.563628    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:55.909291 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:55.909257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:56.225611 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:56.225503    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:56.326102 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:56.326072    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:58.891149 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.891109    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:58.969815 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.969775    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:58.970033 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970010    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"]
Jul 19 14:49:58.970389 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970377    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6700eb65-6167-410d-8493-bbda36d7b573" containerName="maas-api"
Jul 19 14:49:58.970430 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970391    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6700eb65-6167-410d-8493-bbda36d7b573" containerName="maas-api"
Jul 19 14:49:58.970430 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970400    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e1dc97-85be-4b6f-ae39-015e7267d2ac" containerName="maas-api"
Jul 19 14:49:58.970430 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970405    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e1dc97-85be-4b6f-ae39-015e7267d2ac" containerName="maas-api"
Jul 19 14:49:58.970551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970468    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6700eb65-6167-410d-8493-bbda36d7b573" containerName="maas-api"
Jul 19 14:49:58.970551 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.970476    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e1dc97-85be-4b6f-ae39-015e7267d2ac" containerName="maas-api"
Jul 19 14:49:58.980026 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.980001    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:58.982619 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.982597    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-3caf77-serving-cert\""
Jul 19 14:49:58.988738 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:58.988716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"]
Jul 19 14:49:59.035133 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.035106    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:59.118491 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.118468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.118608 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.118567    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvkp5\" (UniqueName: \"kubernetes.io/projected/55a97167-9b79-4317-a07d-d5fe8b965daa-kube-api-access-cvkp5\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.118608 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.118587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a97167-9b79-4317-a07d-d5fe8b965daa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.219782 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.219717    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvkp5\" (UniqueName: \"kubernetes.io/projected/55a97167-9b79-4317-a07d-d5fe8b965daa-kube-api-access-cvkp5\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.219782 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.219746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a97167-9b79-4317-a07d-d5fe8b965daa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.219782 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.219775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.219952 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:49:59.219884    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-serving-cert: secret "maas-api-e2e-shared-a-3caf77-serving-cert" not found
Jul 19 14:49:59.219984 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:49:59.219953    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls podName:55a97167-9b79-4317-a07d-d5fe8b965daa nodeName:}" failed. No retries permitted until 2026-07-19 14:49:59.719925793 +0000 UTC m=+2218.022756787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls") pod "maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" (UID: "55a97167-9b79-4317-a07d-d5fe8b965daa") : secret "maas-api-e2e-shared-a-3caf77-serving-cert" not found
Jul 19 14:49:59.220461 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.220438    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a97167-9b79-4317-a07d-d5fe8b965daa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.232345 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.232322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvkp5\" (UniqueName: \"kubernetes.io/projected/55a97167-9b79-4317-a07d-d5fe8b965daa-kube-api-access-cvkp5\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.337920 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.337890    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:59.533385 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.533295    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:59.631673 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.631642    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:59.724696 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.724666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.726910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.726893    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:49:59.894386 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:49:59.894350    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:50:00.236631 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:00.236604    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"]
Jul 19 14:50:00.238113 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:50:00.238084    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a97167_9b79_4317_a07d_d5fe8b965daa.slice/crio-f3a7800f036173a244ddffc853cf2f19f36ceb30ae006baa322e8b99b58c2531 WatchSource:0}: Error finding container f3a7800f036173a244ddffc853cf2f19f36ceb30ae006baa322e8b99b58c2531: Status 404 returned error can't find the container with id f3a7800f036173a244ddffc853cf2f19f36ceb30ae006baa322e8b99b58c2531
Jul 19 14:50:00.239742 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:00.239724    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:50:00.478559 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:00.478505    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" event={"ID":"55a97167-9b79-4317-a07d-d5fe8b965daa","Type":"ContainerStarted","Data":"f3a7800f036173a244ddffc853cf2f19f36ceb30ae006baa322e8b99b58c2531"}
Jul 19 14:50:01.485448 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:01.485414    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" event={"ID":"55a97167-9b79-4317-a07d-d5fe8b965daa","Type":"ContainerStarted","Data":"7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5"}
Jul 19 14:50:01.485937 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:01.485556    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:50:01.500707 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:01.500665    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" podStartSLOduration=3.094330453 podStartE2EDuration="3.500651715s" totalImagesPullingTime="406.321262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:50:01.499791093 +0000 UTC m=+2219.802622109" watchObservedRunningTime="2026-07-19 14:50:01.500651715 +0000 UTC m=+2219.803482731"
Jul 19 14:50:03.400422 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:03.400396    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:03.733238 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:03.733153    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:04.027106 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:04.027028    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:04.331736 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:04.331625    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:06.953607 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:06.953568    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:07.053478 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.053450    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:07.104936 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.104908    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"]
Jul 19 14:50:07.108226 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.108210    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.110876 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.110858    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-adad07-serving-cert\""
Jul 19 14:50:07.120841 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.120819    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"]
Jul 19 14:50:07.153353 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.153305    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:07.187676 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.187648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682d6692-7953-4256-86b0-f0e3ba523084-maas-api-tls\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.187832 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.187676    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks24m\" (UniqueName: \"kubernetes.io/projected/682d6692-7953-4256-86b0-f0e3ba523084-kube-api-access-ks24m\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.187832 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.187724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682d6692-7953-4256-86b0-f0e3ba523084-openshift-service-ca\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.288310 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.288232    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682d6692-7953-4256-86b0-f0e3ba523084-maas-api-tls\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.288310 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.288270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks24m\" (UniqueName: \"kubernetes.io/projected/682d6692-7953-4256-86b0-f0e3ba523084-kube-api-access-ks24m\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.288310 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.288303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682d6692-7953-4256-86b0-f0e3ba523084-openshift-service-ca\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.289064 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.289040    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682d6692-7953-4256-86b0-f0e3ba523084-openshift-service-ca\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.290531 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.290503    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682d6692-7953-4256-86b0-f0e3ba523084-maas-api-tls\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.304896 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.304874    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks24m\" (UniqueName: \"kubernetes.io/projected/682d6692-7953-4256-86b0-f0e3ba523084-kube-api-access-ks24m\") pod \"maas-api-e2e-shared-b-adad07-9c49cd566-7znxt\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.410091 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.410050    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:07.422651 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.422630    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:07.500180 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.500148    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:50:07.581082 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.581051    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"]
Jul 19 14:50:07.582771 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:50:07.582725    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod682d6692_7953_4256_86b0_f0e3ba523084.slice/crio-345d5aa351de63d8fe2dab481ac5709646f883f9ffa2a37ecdfdc2d71e8a64aa WatchSource:0}: Error finding container 345d5aa351de63d8fe2dab481ac5709646f883f9ffa2a37ecdfdc2d71e8a64aa: Status 404 returned error can't find the container with id 345d5aa351de63d8fe2dab481ac5709646f883f9ffa2a37ecdfdc2d71e8a64aa
Jul 19 14:50:07.717576 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.717541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:07.818748 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:07.818710    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:08.518596 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:08.518559    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt" event={"ID":"682d6692-7953-4256-86b0-f0e3ba523084","Type":"ContainerStarted","Data":"5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0"}
Jul 19 14:50:08.519023 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:08.518618    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt" event={"ID":"682d6692-7953-4256-86b0-f0e3ba523084","Type":"ContainerStarted","Data":"345d5aa351de63d8fe2dab481ac5709646f883f9ffa2a37ecdfdc2d71e8a64aa"}
Jul 19 14:50:08.519023 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:08.518686    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:08.534410 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:08.534369    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt" podStartSLOduration=1.233146856 podStartE2EDuration="1.534356163s" totalImagesPullingTime="301.209307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:50:08.532734658 +0000 UTC m=+2226.835565675" watchObservedRunningTime="2026-07-19 14:50:08.534356163 +0000 UTC m=+2226.837187178"
Jul 19 14:50:11.434392 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:11.434361    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:14.528926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:14.528899    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:50:20.972855 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:20.972809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:21.072628 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:21.072597    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:27.787916 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:27.787839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:28.438027 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:28.438000    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:28.540124 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:28.540089    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:45.707567 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:50:45.707507    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:46.937361 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:51:46.937317    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:47.042338 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:51:47.042299    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:02.837802 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:52:02.837720    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:13.493844 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:52:13.493804    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:40.856417 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:52:40.856372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:41.633161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:52:41.633130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:48.621947 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:52:48.621911    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:02.223534 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:53:02.223493    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:53:02.233560 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:53:02.233513    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h4r72_c6f99107-5ce9-4e02-925d-00c443973ba2/ovn-acl-logging/0.log"
Jul 19 14:53:15.027630 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:53:15.027593    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:00.392579 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:00.392517    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:01.581470 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:01.581436    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:02.358577 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:02.358546    2574 scope.go:117] "RemoveContainer" containerID="8d5c45d14faa7f475f51208670cf87d2eb63d4b7efbefbf5409c6d88cecec890"
Jul 19 14:54:15.236296 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:15.236265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:25.632995 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:25.632961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:25.681496 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:25.681465    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:40.804207 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:54:40.804170    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:18.303779 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:18.303697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:19.231210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:19.231174    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:20.177723 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.177692    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7520-140503874893744"]
Jul 19 14:55:20.181135 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.181120    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:20.183731 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.183707    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-n7zsk\""
Jul 19 14:55:20.184219 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.184201    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874893744"]
Jul 19 14:55:20.208426 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.208398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg2x5\" (UniqueName: \"kubernetes.io/projected/74bd3c94-cab1-4f3a-af3e-707e99743572-kube-api-access-dg2x5\") pod \"test-curl-7520-140503874893744\" (UID: \"74bd3c94-cab1-4f3a-af3e-707e99743572\") " pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:20.309817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.309791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg2x5\" (UniqueName: \"kubernetes.io/projected/74bd3c94-cab1-4f3a-af3e-707e99743572-kube-api-access-dg2x5\") pod \"test-curl-7520-140503874893744\" (UID: \"74bd3c94-cab1-4f3a-af3e-707e99743572\") " pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:20.317182 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.317159    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg2x5\" (UniqueName: \"kubernetes.io/projected/74bd3c94-cab1-4f3a-af3e-707e99743572-kube-api-access-dg2x5\") pod \"test-curl-7520-140503874893744\" (UID: \"74bd3c94-cab1-4f3a-af3e-707e99743572\") " pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:20.505360 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.505284    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:20.627171 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.627142    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874893744"]
Jul 19 14:55:20.629681 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:55:20.629654    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74bd3c94_cab1_4f3a_af3e_707e99743572.slice/crio-64cb710d29c171acc0a0f15a99f46cdb339e01d097232432dc87347a40a4eaa2 WatchSource:0}: Error finding container 64cb710d29c171acc0a0f15a99f46cdb339e01d097232432dc87347a40a4eaa2: Status 404 returned error can't find the container with id 64cb710d29c171acc0a0f15a99f46cdb339e01d097232432dc87347a40a4eaa2
Jul 19 14:55:20.630910 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.630894    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:55:20.982991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:20.982944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874893744" event={"ID":"74bd3c94-cab1-4f3a-af3e-707e99743572","Type":"ContainerStarted","Data":"64cb710d29c171acc0a0f15a99f46cdb339e01d097232432dc87347a40a4eaa2"}
Jul 19 14:55:21.722609 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:21.722577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:21.824945 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:21.824916    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:21.989780 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:21.989710    2574 generic.go:356] "Generic (PLEG): container finished" podID="74bd3c94-cab1-4f3a-af3e-707e99743572" containerID="b82f2836896e226b47df1768c8156f4420b803d75b9beab7a732482a357fde6a" exitCode=0
Jul 19 14:55:21.989982 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:21.989780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874893744" event={"ID":"74bd3c94-cab1-4f3a-af3e-707e99743572","Type":"ContainerDied","Data":"b82f2836896e226b47df1768c8156f4420b803d75b9beab7a732482a357fde6a"}
Jul 19 14:55:23.120751 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.120724    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:23.142007 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.141980    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7520-140503874893744_74bd3c94-cab1-4f3a-af3e-707e99743572/test-curl-7520-140503874893744/0.log"
Jul 19 14:55:23.167897 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.167866    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7520-140503874893744"]
Jul 19 14:55:23.172237 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.172209    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7520-140503874893744"]
Jul 19 14:55:23.233173 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.233148    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg2x5\" (UniqueName: \"kubernetes.io/projected/74bd3c94-cab1-4f3a-af3e-707e99743572-kube-api-access-dg2x5\") pod \"74bd3c94-cab1-4f3a-af3e-707e99743572\" (UID: \"74bd3c94-cab1-4f3a-af3e-707e99743572\") "
Jul 19 14:55:23.235161 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.235139    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74bd3c94-cab1-4f3a-af3e-707e99743572-kube-api-access-dg2x5" (OuterVolumeSpecName: "kube-api-access-dg2x5") pod "74bd3c94-cab1-4f3a-af3e-707e99743572" (UID: "74bd3c94-cab1-4f3a-af3e-707e99743572"). InnerVolumeSpecName "kube-api-access-dg2x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:23.292002 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.291931    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7520-140503874895536"]
Jul 19 14:55:23.292321 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.292309    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74bd3c94-cab1-4f3a-af3e-707e99743572" containerName="test-curl-7520-140503874893744"
Jul 19 14:55:23.292364 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.292322    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="74bd3c94-cab1-4f3a-af3e-707e99743572" containerName="test-curl-7520-140503874893744"
Jul 19 14:55:23.292397 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.292389    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="74bd3c94-cab1-4f3a-af3e-707e99743572" containerName="test-curl-7520-140503874893744"
Jul 19 14:55:23.295482 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.295466    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:23.298238 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.298201    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874895536"]
Jul 19 14:55:23.334285 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.334249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q72m5\" (UniqueName: \"kubernetes.io/projected/ea370ab6-6933-472d-b163-9f6f5f2b0b7e-kube-api-access-q72m5\") pod \"test-curl-7520-140503874895536\" (UID: \"ea370ab6-6933-472d-b163-9f6f5f2b0b7e\") " pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:23.334416 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.334304    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg2x5\" (UniqueName: \"kubernetes.io/projected/74bd3c94-cab1-4f3a-af3e-707e99743572-kube-api-access-dg2x5\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:23.435540 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.435486    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q72m5\" (UniqueName: \"kubernetes.io/projected/ea370ab6-6933-472d-b163-9f6f5f2b0b7e-kube-api-access-q72m5\") pod \"test-curl-7520-140503874895536\" (UID: \"ea370ab6-6933-472d-b163-9f6f5f2b0b7e\") " pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:23.442803 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.442774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q72m5\" (UniqueName: \"kubernetes.io/projected/ea370ab6-6933-472d-b163-9f6f5f2b0b7e-kube-api-access-q72m5\") pod \"test-curl-7520-140503874895536\" (UID: \"ea370ab6-6933-472d-b163-9f6f5f2b0b7e\") " pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:23.606125 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.606044    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:23.731774 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.731747    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874895536"]
Jul 19 14:55:23.733539 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:55:23.733495    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea370ab6_6933_472d_b163_9f6f5f2b0b7e.slice/crio-12f0cf240cde7b98d8820c9ff4a37a0f3fa96e095e78b6d7cb55cbc41f6afe53 WatchSource:0}: Error finding container 12f0cf240cde7b98d8820c9ff4a37a0f3fa96e095e78b6d7cb55cbc41f6afe53: Status 404 returned error can't find the container with id 12f0cf240cde7b98d8820c9ff4a37a0f3fa96e095e78b6d7cb55cbc41f6afe53
Jul 19 14:55:23.999294 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.999269    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:23.999457 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:23.999311    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64cb710d29c171acc0a0f15a99f46cdb339e01d097232432dc87347a40a4eaa2"
Jul 19 14:55:24.000472 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:24.000450    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874895536" event={"ID":"ea370ab6-6933-472d-b163-9f6f5f2b0b7e","Type":"ContainerStarted","Data":"12f0cf240cde7b98d8820c9ff4a37a0f3fa96e095e78b6d7cb55cbc41f6afe53"}
Jul 19 14:55:24.009451 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:55:24.009424    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7520-140503874893744\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="74bd3c94-cab1-4f3a-af3e-707e99743572" pod="opendatahub/test-curl-7520-140503874893744"
Jul 19 14:55:24.396904 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:24.396869    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74bd3c94-cab1-4f3a-af3e-707e99743572" path="/var/lib/kubelet/pods/74bd3c94-cab1-4f3a-af3e-707e99743572/volumes"
Jul 19 14:55:25.006342 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:25.006304    2574 generic.go:356] "Generic (PLEG): container finished" podID="ea370ab6-6933-472d-b163-9f6f5f2b0b7e" containerID="682527eafa88b6a11c0409aacc1219f847277924f5fc98e774858c4c69db2bcc" exitCode=0
Jul 19 14:55:25.006495 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:25.006393    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874895536" event={"ID":"ea370ab6-6933-472d-b163-9f6f5f2b0b7e","Type":"ContainerDied","Data":"682527eafa88b6a11c0409aacc1219f847277924f5fc98e774858c4c69db2bcc"}
Jul 19 14:55:26.133471 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.133449    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:26.152672 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.152648    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7520-140503874895536_ea370ab6-6933-472d-b163-9f6f5f2b0b7e/test-curl-7520-140503874895536/0.log"
Jul 19 14:55:26.176629 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.176605    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7520-140503874895536"]
Jul 19 14:55:26.181085 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.181065    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7520-140503874895536"]
Jul 19 14:55:26.257621 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.257599    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q72m5\" (UniqueName: \"kubernetes.io/projected/ea370ab6-6933-472d-b163-9f6f5f2b0b7e-kube-api-access-q72m5\") pod \"ea370ab6-6933-472d-b163-9f6f5f2b0b7e\" (UID: \"ea370ab6-6933-472d-b163-9f6f5f2b0b7e\") "
Jul 19 14:55:26.259548 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.259507    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea370ab6-6933-472d-b163-9f6f5f2b0b7e-kube-api-access-q72m5" (OuterVolumeSpecName: "kube-api-access-q72m5") pod "ea370ab6-6933-472d-b163-9f6f5f2b0b7e" (UID: "ea370ab6-6933-472d-b163-9f6f5f2b0b7e"). InnerVolumeSpecName "kube-api-access-q72m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:26.344975 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.344945    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:26.345303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.345290    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea370ab6-6933-472d-b163-9f6f5f2b0b7e" containerName="test-curl-7520-140503874895536"
Jul 19 14:55:26.345345 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.345304    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea370ab6-6933-472d-b163-9f6f5f2b0b7e" containerName="test-curl-7520-140503874895536"
Jul 19 14:55:26.345378 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.345371    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea370ab6-6933-472d-b163-9f6f5f2b0b7e" containerName="test-curl-7520-140503874895536"
Jul 19 14:55:26.351592 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.349824    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:26.355398 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.355373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:26.358195 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.358173    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q72m5\" (UniqueName: \"kubernetes.io/projected/ea370ab6-6933-472d-b163-9f6f5f2b0b7e-kube-api-access-q72m5\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:26.396456 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.396426    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea370ab6-6933-472d-b163-9f6f5f2b0b7e" path="/var/lib/kubelet/pods/ea370ab6-6933-472d-b163-9f6f5f2b0b7e/volumes"
Jul 19 14:55:26.458893 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.458869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq6g6\" (UniqueName: \"kubernetes.io/projected/e9850be9-4133-4268-b40d-d526558ce475-kube-api-access-fq6g6\") pod \"test-curl-7520-140503874868400\" (UID: \"e9850be9-4133-4268-b40d-d526558ce475\") " pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:26.560089 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.560036    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq6g6\" (UniqueName: \"kubernetes.io/projected/e9850be9-4133-4268-b40d-d526558ce475-kube-api-access-fq6g6\") pod \"test-curl-7520-140503874868400\" (UID: \"e9850be9-4133-4268-b40d-d526558ce475\") " pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:26.568498 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.568478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq6g6\" (UniqueName: \"kubernetes.io/projected/e9850be9-4133-4268-b40d-d526558ce475-kube-api-access-fq6g6\") pod \"test-curl-7520-140503874868400\" (UID: \"e9850be9-4133-4268-b40d-d526558ce475\") " pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:26.660227 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.660202    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:26.784588 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:26.784566    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:26.786466 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:55:26.786440    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9850be9_4133_4268_b40d_d526558ce475.slice/crio-1517efc15748794fedb082dc1148bf1afe1fefcb04cd0bcb1f6581983e4ebbb6 WatchSource:0}: Error finding container 1517efc15748794fedb082dc1148bf1afe1fefcb04cd0bcb1f6581983e4ebbb6: Status 404 returned error can't find the container with id 1517efc15748794fedb082dc1148bf1afe1fefcb04cd0bcb1f6581983e4ebbb6
Jul 19 14:55:27.015978 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:27.015945    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874868400" event={"ID":"e9850be9-4133-4268-b40d-d526558ce475","Type":"ContainerStarted","Data":"1517efc15748794fedb082dc1148bf1afe1fefcb04cd0bcb1f6581983e4ebbb6"}
Jul 19 14:55:27.017115 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:27.017093    2574 scope.go:117] "RemoveContainer" containerID="682527eafa88b6a11c0409aacc1219f847277924f5fc98e774858c4c69db2bcc"
Jul 19 14:55:27.017220 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:27.017099    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874895536"
Jul 19 14:55:27.252829 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:27.252799    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:28.023108 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:28.023073    2574 generic.go:356] "Generic (PLEG): container finished" podID="e9850be9-4133-4268-b40d-d526558ce475" containerID="9ea6a2318d36f7bf0570431b9e050fb831f72309f258cd0c162e388cae49e284" exitCode=0
Jul 19 14:55:28.023268 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:28.023139    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874868400" event={"ID":"e9850be9-4133-4268-b40d-d526558ce475","Type":"ContainerDied","Data":"9ea6a2318d36f7bf0570431b9e050fb831f72309f258cd0c162e388cae49e284"}
Jul 19 14:55:29.184861 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.184836    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:29.205316 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.205283    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7520-140503874868400_e9850be9-4133-4268-b40d-d526558ce475/test-curl-7520-140503874868400/0.log"
Jul 19 14:55:29.230377 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.230339    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:29.234959 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.234935    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:29.281335 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.281258    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq6g6\" (UniqueName: \"kubernetes.io/projected/e9850be9-4133-4268-b40d-d526558ce475-kube-api-access-fq6g6\") pod \"e9850be9-4133-4268-b40d-d526558ce475\" (UID: \"e9850be9-4133-4268-b40d-d526558ce475\") "
Jul 19 14:55:29.283356 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.283331    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9850be9-4133-4268-b40d-d526558ce475-kube-api-access-fq6g6" (OuterVolumeSpecName: "kube-api-access-fq6g6") pod "e9850be9-4133-4268-b40d-d526558ce475" (UID: "e9850be9-4133-4268-b40d-d526558ce475"). InnerVolumeSpecName "kube-api-access-fq6g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:29.343834 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.343801    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:29.354726 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.354701    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:29.355066 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.355054    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9850be9-4133-4268-b40d-d526558ce475" containerName="test-curl-7520-140503874868400"
Jul 19 14:55:29.355109 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.355068    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9850be9-4133-4268-b40d-d526558ce475" containerName="test-curl-7520-140503874868400"
Jul 19 14:55:29.355146 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.355137    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9850be9-4133-4268-b40d-d526558ce475" containerName="test-curl-7520-140503874868400"
Jul 19 14:55:29.358295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.358275    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:29.364151 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.364124    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:29.382039 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.382012    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq6g6\" (UniqueName: \"kubernetes.io/projected/e9850be9-4133-4268-b40d-d526558ce475-kube-api-access-fq6g6\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:29.444173 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.444121    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:29.482600 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.482570    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s4wl\" (UniqueName: \"kubernetes.io/projected/bccb4ae5-8c93-47fa-858d-c43e25e09f88-kube-api-access-8s4wl\") pod \"test-curl-7520-140503874868400\" (UID: \"bccb4ae5-8c93-47fa-858d-c43e25e09f88\") " pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:29.584028 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.583947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s4wl\" (UniqueName: \"kubernetes.io/projected/bccb4ae5-8c93-47fa-858d-c43e25e09f88-kube-api-access-8s4wl\") pod \"test-curl-7520-140503874868400\" (UID: \"bccb4ae5-8c93-47fa-858d-c43e25e09f88\") " pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:29.591658 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.591624    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s4wl\" (UniqueName: \"kubernetes.io/projected/bccb4ae5-8c93-47fa-858d-c43e25e09f88-kube-api-access-8s4wl\") pod \"test-curl-7520-140503874868400\" (UID: \"bccb4ae5-8c93-47fa-858d-c43e25e09f88\") " pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:29.668567 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.668538    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:29.796968 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:29.796944    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:29.799096 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:55:29.799067    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbccb4ae5_8c93_47fa_858d_c43e25e09f88.slice/crio-a84451ea3c665bf1d25e7910443512c6f863629369a52543de9030481b2ee656 WatchSource:0}: Error finding container a84451ea3c665bf1d25e7910443512c6f863629369a52543de9030481b2ee656: Status 404 returned error can't find the container with id a84451ea3c665bf1d25e7910443512c6f863629369a52543de9030481b2ee656
Jul 19 14:55:30.034013 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:30.033976    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:30.034186 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:30.033990    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1517efc15748794fedb082dc1148bf1afe1fefcb04cd0bcb1f6581983e4ebbb6"
Jul 19 14:55:30.035181 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:30.035158    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874868400" event={"ID":"bccb4ae5-8c93-47fa-858d-c43e25e09f88","Type":"ContainerStarted","Data":"a84451ea3c665bf1d25e7910443512c6f863629369a52543de9030481b2ee656"}
Jul 19 14:55:30.044590 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:30.044562    2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7520-140503874868400" oldPodUID="e9850be9-4133-4268-b40d-d526558ce475" podUID="bccb4ae5-8c93-47fa-858d-c43e25e09f88"
Jul 19 14:55:30.397582 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:30.397504    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9850be9-4133-4268-b40d-d526558ce475" path="/var/lib/kubelet/pods/e9850be9-4133-4268-b40d-d526558ce475/volumes"
Jul 19 14:55:31.041558 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:31.041506    2574 generic.go:356] "Generic (PLEG): container finished" podID="bccb4ae5-8c93-47fa-858d-c43e25e09f88" containerID="4b54d818074c3494c68c2cbfb8aa1cef33ad616e64cb53ca244f7bf744ed81ed" exitCode=0
Jul 19 14:55:31.041767 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:31.041579    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874868400" event={"ID":"bccb4ae5-8c93-47fa-858d-c43e25e09f88","Type":"ContainerDied","Data":"4b54d818074c3494c68c2cbfb8aa1cef33ad616e64cb53ca244f7bf744ed81ed"}
Jul 19 14:55:32.175394 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.175367    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:32.193585 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.193558    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7520-140503874868400_bccb4ae5-8c93-47fa-858d-c43e25e09f88/test-curl-7520-140503874868400/0.log"
Jul 19 14:55:32.218733 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.218705    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:32.221934 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.221912    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7520-140503874868400"]
Jul 19 14:55:32.307860 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.307774    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s4wl\" (UniqueName: \"kubernetes.io/projected/bccb4ae5-8c93-47fa-858d-c43e25e09f88-kube-api-access-8s4wl\") pod \"bccb4ae5-8c93-47fa-858d-c43e25e09f88\" (UID: \"bccb4ae5-8c93-47fa-858d-c43e25e09f88\") "
Jul 19 14:55:32.309813 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.309788    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccb4ae5-8c93-47fa-858d-c43e25e09f88-kube-api-access-8s4wl" (OuterVolumeSpecName: "kube-api-access-8s4wl") pod "bccb4ae5-8c93-47fa-858d-c43e25e09f88" (UID: "bccb4ae5-8c93-47fa-858d-c43e25e09f88"). InnerVolumeSpecName "kube-api-access-8s4wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:32.343976 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.343936    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7520-140503874869408"]
Jul 19 14:55:32.344407 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.344389    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccb4ae5-8c93-47fa-858d-c43e25e09f88" containerName="test-curl-7520-140503874868400"
Jul 19 14:55:32.344464 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.344410    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccb4ae5-8c93-47fa-858d-c43e25e09f88" containerName="test-curl-7520-140503874868400"
Jul 19 14:55:32.344500 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.344474    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccb4ae5-8c93-47fa-858d-c43e25e09f88" containerName="test-curl-7520-140503874868400"
Jul 19 14:55:32.347341 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.347327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:32.353202 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.353176    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874869408"]
Jul 19 14:55:32.397271 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.397235    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bccb4ae5-8c93-47fa-858d-c43e25e09f88" path="/var/lib/kubelet/pods/bccb4ae5-8c93-47fa-858d-c43e25e09f88/volumes"
Jul 19 14:55:32.409208 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.409184    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzlmn\" (UniqueName: \"kubernetes.io/projected/21cfd2d1-3368-414b-98c8-621d54b02d44-kube-api-access-nzlmn\") pod \"test-curl-7520-140503874869408\" (UID: \"21cfd2d1-3368-414b-98c8-621d54b02d44\") " pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:32.409303 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.409253    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s4wl\" (UniqueName: \"kubernetes.io/projected/bccb4ae5-8c93-47fa-858d-c43e25e09f88-kube-api-access-8s4wl\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:32.510388 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.510357    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzlmn\" (UniqueName: \"kubernetes.io/projected/21cfd2d1-3368-414b-98c8-621d54b02d44-kube-api-access-nzlmn\") pod \"test-curl-7520-140503874869408\" (UID: \"21cfd2d1-3368-414b-98c8-621d54b02d44\") " pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:32.519606 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.519581    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzlmn\" (UniqueName: \"kubernetes.io/projected/21cfd2d1-3368-414b-98c8-621d54b02d44-kube-api-access-nzlmn\") pod \"test-curl-7520-140503874869408\" (UID: \"21cfd2d1-3368-414b-98c8-621d54b02d44\") " pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:32.658544 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.658489    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:32.788178 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:32.788155    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7520-140503874869408"]
Jul 19 14:55:32.790240 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:55:32.790212    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21cfd2d1_3368_414b_98c8_621d54b02d44.slice/crio-0a0d8586936d70174d9a0832be7d57cbed66abc301c0561d892f77dd600af5cf WatchSource:0}: Error finding container 0a0d8586936d70174d9a0832be7d57cbed66abc301c0561d892f77dd600af5cf: Status 404 returned error can't find the container with id 0a0d8586936d70174d9a0832be7d57cbed66abc301c0561d892f77dd600af5cf
Jul 19 14:55:33.051820 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:33.051749    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874868400"
Jul 19 14:55:33.051979 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:33.051757    2574 scope.go:117] "RemoveContainer" containerID="4b54d818074c3494c68c2cbfb8aa1cef33ad616e64cb53ca244f7bf744ed81ed"
Jul 19 14:55:33.053177 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:33.053094    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874869408" event={"ID":"21cfd2d1-3368-414b-98c8-621d54b02d44","Type":"ContainerStarted","Data":"0a0d8586936d70174d9a0832be7d57cbed66abc301c0561d892f77dd600af5cf"}
Jul 19 14:55:34.062963 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:34.062931    2574 generic.go:356] "Generic (PLEG): container finished" podID="21cfd2d1-3368-414b-98c8-621d54b02d44" containerID="bf1b73b6a129a3a13ab886ec5c378843807edf28472b56f3b483ded5fb1705a9" exitCode=0
Jul 19 14:55:34.063329 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:34.063006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7520-140503874869408" event={"ID":"21cfd2d1-3368-414b-98c8-621d54b02d44","Type":"ContainerDied","Data":"bf1b73b6a129a3a13ab886ec5c378843807edf28472b56f3b483ded5fb1705a9"}
Jul 19 14:55:35.201801 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.201777    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:35.229570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.229537    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7520-140503874869408_21cfd2d1-3368-414b-98c8-621d54b02d44/test-curl-7520-140503874869408/0.log"
Jul 19 14:55:35.253463 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.253422    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7520-140503874869408"]
Jul 19 14:55:35.262210 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.259037    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7520-140503874869408"]
Jul 19 14:55:35.336584 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.336499    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzlmn\" (UniqueName: \"kubernetes.io/projected/21cfd2d1-3368-414b-98c8-621d54b02d44-kube-api-access-nzlmn\") pod \"21cfd2d1-3368-414b-98c8-621d54b02d44\" (UID: \"21cfd2d1-3368-414b-98c8-621d54b02d44\") "
Jul 19 14:55:35.338570 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.338537    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21cfd2d1-3368-414b-98c8-621d54b02d44-kube-api-access-nzlmn" (OuterVolumeSpecName: "kube-api-access-nzlmn") pod "21cfd2d1-3368-414b-98c8-621d54b02d44" (UID: "21cfd2d1-3368-414b-98c8-621d54b02d44"). InnerVolumeSpecName "kube-api-access-nzlmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:35.437424 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:35.437386    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzlmn\" (UniqueName: \"kubernetes.io/projected/21cfd2d1-3368-414b-98c8-621d54b02d44-kube-api-access-nzlmn\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:36.074431 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:36.074402    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a0d8586936d70174d9a0832be7d57cbed66abc301c0561d892f77dd600af5cf"
Jul 19 14:55:36.074431 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:36.074414    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:36.083926 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:55:36.083897    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7520-140503874869408\" is forbidden: User \"system:node:ip-10-0-143-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-90.ec2.internal' and this object" podUID="21cfd2d1-3368-414b-98c8-621d54b02d44" pod="opendatahub/test-curl-7520-140503874869408"
Jul 19 14:55:36.396443 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:36.396401    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21cfd2d1-3368-414b-98c8-621d54b02d44" path="/var/lib/kubelet/pods/21cfd2d1-3368-414b-98c8-621d54b02d44/volumes"
Jul 19 14:55:38.921335 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:38.921296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:39.061218 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:39.061186    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:49.123012 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.122979    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"]
Jul 19 14:55:49.123396 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.123208    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" podUID="55a97167-9b79-4317-a07d-d5fe8b965daa" containerName="maas-api" containerID="cri-o://7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5" gracePeriod=30
Jul 19 14:55:49.166490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.166454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:49.367793 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.367769    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:55:49.464943 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.464869    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls\") pod \"55a97167-9b79-4317-a07d-d5fe8b965daa\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") "
Jul 19 14:55:49.464943 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.464923    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a97167-9b79-4317-a07d-d5fe8b965daa-openshift-service-ca\") pod \"55a97167-9b79-4317-a07d-d5fe8b965daa\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") "
Jul 19 14:55:49.465173 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.464962    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvkp5\" (UniqueName: \"kubernetes.io/projected/55a97167-9b79-4317-a07d-d5fe8b965daa-kube-api-access-cvkp5\") pod \"55a97167-9b79-4317-a07d-d5fe8b965daa\" (UID: \"55a97167-9b79-4317-a07d-d5fe8b965daa\") "
Jul 19 14:55:49.465381 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.465346    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55a97167-9b79-4317-a07d-d5fe8b965daa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55a97167-9b79-4317-a07d-d5fe8b965daa" (UID: "55a97167-9b79-4317-a07d-d5fe8b965daa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:55:49.467046 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.467024    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a97167-9b79-4317-a07d-d5fe8b965daa-kube-api-access-cvkp5" (OuterVolumeSpecName: "kube-api-access-cvkp5") pod "55a97167-9b79-4317-a07d-d5fe8b965daa" (UID: "55a97167-9b79-4317-a07d-d5fe8b965daa"). InnerVolumeSpecName "kube-api-access-cvkp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:49.467125 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.467063    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55a97167-9b79-4317-a07d-d5fe8b965daa" (UID: "55a97167-9b79-4317-a07d-d5fe8b965daa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:55:49.566564 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.566509    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55a97167-9b79-4317-a07d-d5fe8b965daa-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:49.566564 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.566561    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55a97167-9b79-4317-a07d-d5fe8b965daa-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:49.566564 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:49.566571    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvkp5\" (UniqueName: \"kubernetes.io/projected/55a97167-9b79-4317-a07d-d5fe8b965daa-kube-api-access-cvkp5\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:55:50.141746 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.141714    2574 generic.go:356] "Generic (PLEG): container finished" podID="55a97167-9b79-4317-a07d-d5fe8b965daa" containerID="7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5" exitCode=0
Jul 19 14:55:50.142211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.141753    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"
Jul 19 14:55:50.142211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.141777    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" event={"ID":"55a97167-9b79-4317-a07d-d5fe8b965daa","Type":"ContainerDied","Data":"7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5"}
Jul 19 14:55:50.142211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.141828    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc" event={"ID":"55a97167-9b79-4317-a07d-d5fe8b965daa","Type":"ContainerDied","Data":"f3a7800f036173a244ddffc853cf2f19f36ceb30ae006baa322e8b99b58c2531"}
Jul 19 14:55:50.142211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.141860    2574 scope.go:117] "RemoveContainer" containerID="7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5"
Jul 19 14:55:50.150694 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.150674    2574 scope.go:117] "RemoveContainer" containerID="7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5"
Jul 19 14:55:50.150915 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:55:50.150897    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5\": container with ID starting with 7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5 not found: ID does not exist" containerID="7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5"
Jul 19 14:55:50.150969 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.150923    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5"} err="failed to get container status \"7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5\": rpc error: code = NotFound desc = could not find container \"7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5\": container with ID starting with 7c3d28813a4b8c1bc1c86896213a833f8ae1cadd3390426f942cd0d34d7beff5 not found: ID does not exist"
Jul 19 14:55:50.166877 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.166851    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"]
Jul 19 14:55:50.172906 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.172883    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3caf77-76bc79464c-hshmc"]
Jul 19 14:55:50.397181 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:50.397090    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a97167-9b79-4317-a07d-d5fe8b965daa" path="/var/lib/kubelet/pods/55a97167-9b79-4317-a07d-d5fe8b965daa/volumes"
Jul 19 14:55:53.444937 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:53.444895    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:53.854176 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:55:53.854085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:00.182076 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:00.182031    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:00.325070 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:00.325034    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:00.529007 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:00.528917    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:00.779396 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:00.779307    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:01.780141 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:01.780102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:01.896394 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:01.896358    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:11.435552 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.435502    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"]
Jul 19 14:56:11.436328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.436082    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt" podUID="682d6692-7953-4256-86b0-f0e3ba523084" containerName="maas-api" containerID="cri-o://5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0" gracePeriod=30
Jul 19 14:56:11.460944 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.460922    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:11.681755 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.681727    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:56:11.759894 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.759819    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682d6692-7953-4256-86b0-f0e3ba523084-openshift-service-ca\") pod \"682d6692-7953-4256-86b0-f0e3ba523084\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") "
Jul 19 14:56:11.760034 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.759909    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks24m\" (UniqueName: \"kubernetes.io/projected/682d6692-7953-4256-86b0-f0e3ba523084-kube-api-access-ks24m\") pod \"682d6692-7953-4256-86b0-f0e3ba523084\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") "
Jul 19 14:56:11.760034 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.759941    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682d6692-7953-4256-86b0-f0e3ba523084-maas-api-tls\") pod \"682d6692-7953-4256-86b0-f0e3ba523084\" (UID: \"682d6692-7953-4256-86b0-f0e3ba523084\") "
Jul 19 14:56:11.760166 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.760144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/682d6692-7953-4256-86b0-f0e3ba523084-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "682d6692-7953-4256-86b0-f0e3ba523084" (UID: "682d6692-7953-4256-86b0-f0e3ba523084"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:56:11.760269 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.760255    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682d6692-7953-4256-86b0-f0e3ba523084-openshift-service-ca\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:56:11.761869 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.761838    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/682d6692-7953-4256-86b0-f0e3ba523084-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "682d6692-7953-4256-86b0-f0e3ba523084" (UID: "682d6692-7953-4256-86b0-f0e3ba523084"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:56:11.761973 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.761947    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/682d6692-7953-4256-86b0-f0e3ba523084-kube-api-access-ks24m" (OuterVolumeSpecName: "kube-api-access-ks24m") pod "682d6692-7953-4256-86b0-f0e3ba523084" (UID: "682d6692-7953-4256-86b0-f0e3ba523084"). InnerVolumeSpecName "kube-api-access-ks24m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:56:11.860832 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.860804    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks24m\" (UniqueName: \"kubernetes.io/projected/682d6692-7953-4256-86b0-f0e3ba523084-kube-api-access-ks24m\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:56:11.860832 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:11.860830    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682d6692-7953-4256-86b0-f0e3ba523084-maas-api-tls\") on node \"ip-10-0-143-90.ec2.internal\" DevicePath \"\""
Jul 19 14:56:12.246646 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.246617    2574 generic.go:356] "Generic (PLEG): container finished" podID="682d6692-7953-4256-86b0-f0e3ba523084" containerID="5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0" exitCode=0
Jul 19 14:56:12.246817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.246678    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"
Jul 19 14:56:12.246817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.246688    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt" event={"ID":"682d6692-7953-4256-86b0-f0e3ba523084","Type":"ContainerDied","Data":"5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0"}
Jul 19 14:56:12.246817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.246734    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt" event={"ID":"682d6692-7953-4256-86b0-f0e3ba523084","Type":"ContainerDied","Data":"345d5aa351de63d8fe2dab481ac5709646f883f9ffa2a37ecdfdc2d71e8a64aa"}
Jul 19 14:56:12.246817 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.246759    2574 scope.go:117] "RemoveContainer" containerID="5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0"
Jul 19 14:56:12.255746 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.255722    2574 scope.go:117] "RemoveContainer" containerID="5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0"
Jul 19 14:56:12.255994 ip-10-0-143-90 kubenswrapper[2574]: E0719 14:56:12.255971    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0\": container with ID starting with 5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0 not found: ID does not exist" containerID="5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0"
Jul 19 14:56:12.256062 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.256006    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0"} err="failed to get container status \"5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0\": rpc error: code = NotFound desc = could not find container \"5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0\": container with ID starting with 5f81fe1bff38e4088d7fc933fed9ac084304140be69b933b25d13bfa79dc95c0 not found: ID does not exist"
Jul 19 14:56:12.268386 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.268366    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"]
Jul 19 14:56:12.272439 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.272416    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-adad07-9c49cd566-7znxt"]
Jul 19 14:56:12.396811 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:12.396779    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="682d6692-7953-4256-86b0-f0e3ba523084" path="/var/lib/kubelet/pods/682d6692-7953-4256-86b0-f0e3ba523084/volumes"
Jul 19 14:56:16.065632 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:16.065595    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:16.495983 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:16.495947    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:23.431425 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:23.431388    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:23.546516 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:23.546486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:23.778182 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:23.778063    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:24.026312 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:24.026279    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:30.296410 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:30.296327    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zhkx9_78f989a3-2735-4546-b6f2-1c7f82c08499/manager/0.log"
Jul 19 14:56:30.454666 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:30.454638    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57866856f9-ztrk5_3a37e812-ea38-4904-b2b1-995cfac53432/manager/0.log"
Jul 19 14:56:30.844565 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:30.844494    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bd7c6769d-62tfq_8f85a9ec-654f-4c7b-ba7f-95ae715f41bd/copy-manifests/0.log"
Jul 19 14:56:30.868278 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:30.868250    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bd7c6769d-62tfq_8f85a9ec-654f-4c7b-ba7f-95ae715f41bd/manager/0.log"
Jul 19 14:56:31.533702 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:31.533659    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-9b85996-7tvq2_c1e9afaf-db98-4f0a-8a7e-7a631de58076/maas-api/0.log"
Jul 19 14:56:31.912885 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:31.912850    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mxcn8_e16f5aa2-4435-4a66-bb88-72b4d5058da7/postgres/0.log"
Jul 19 14:56:34.909033 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:34.909001    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf8c8f977-grvz4_5379fabd-911c-4adc-ab08-27d4f6367d33/kube-auth-proxy/0.log"
Jul 19 14:56:35.298836 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:35.298743    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jrzjx_841db817-1a2c-4981-99e3-31c9b3d10f98/payload-pre-processing/0.log"
Jul 19 14:56:35.428996 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:35.428968    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ndxvw_79e0cc04-0363-4041-a1e6-aafb1513b74e/payload-processing/0.log"
Jul 19 14:56:35.933691 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:35.933661    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-r9v5z_10f09a39-2f4c-44db-b718-bce84a492038/main/0.log"
Jul 19 14:56:36.357734 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:36.357646    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flhskx_b2d26eae-d1df-4e63-850b-6614ef860290/main/0.log"
Jul 19 14:56:42.932063 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932030    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7f9m7/must-gather-ngsx7"]
Jul 19 14:56:42.932490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932396    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a97167-9b79-4317-a07d-d5fe8b965daa" containerName="maas-api"
Jul 19 14:56:42.932490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932406    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a97167-9b79-4317-a07d-d5fe8b965daa" containerName="maas-api"
Jul 19 14:56:42.932490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932429    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="682d6692-7953-4256-86b0-f0e3ba523084" containerName="maas-api"
Jul 19 14:56:42.932490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932434    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="682d6692-7953-4256-86b0-f0e3ba523084" containerName="maas-api"
Jul 19 14:56:42.932490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932441    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21cfd2d1-3368-414b-98c8-621d54b02d44" containerName="test-curl-7520-140503874869408"
Jul 19 14:56:42.932490 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932446    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21cfd2d1-3368-414b-98c8-621d54b02d44" containerName="test-curl-7520-140503874869408"
Jul 19 14:56:42.932739 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932497    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a97167-9b79-4317-a07d-d5fe8b965daa" containerName="maas-api"
Jul 19 14:56:42.932739 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932506    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="682d6692-7953-4256-86b0-f0e3ba523084" containerName="maas-api"
Jul 19 14:56:42.932739 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.932516    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21cfd2d1-3368-414b-98c8-621d54b02d44" containerName="test-curl-7520-140503874869408"
Jul 19 14:56:42.935653 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.935636    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:42.945826 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.945805    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7f9m7\"/\"default-dockercfg-zm26k\""
Jul 19 14:56:42.945948 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.945932    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7f9m7\"/\"openshift-service-ca.crt\""
Jul 19 14:56:42.946914 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.946900    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7f9m7\"/\"kube-root-ca.crt\""
Jul 19 14:56:42.964262 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.964230    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7f9m7/must-gather-ngsx7"]
Jul 19 14:56:42.982942 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:42.982914    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:43.033377 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.033354    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9fe5a2e-bdf4-4461-a966-d9f47fc786a8-must-gather-output\") pod \"must-gather-ngsx7\" (UID: \"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8\") " pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.033509 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.033424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgpvh\" (UniqueName: \"kubernetes.io/projected/a9fe5a2e-bdf4-4461-a966-d9f47fc786a8-kube-api-access-fgpvh\") pod \"must-gather-ngsx7\" (UID: \"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8\") " pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.120093 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.120061    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:43.134163 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.134134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9fe5a2e-bdf4-4461-a966-d9f47fc786a8-must-gather-output\") pod \"must-gather-ngsx7\" (UID: \"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8\") " pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.134290 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.134213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgpvh\" (UniqueName: \"kubernetes.io/projected/a9fe5a2e-bdf4-4461-a966-d9f47fc786a8-kube-api-access-fgpvh\") pod \"must-gather-ngsx7\" (UID: \"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8\") " pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.134449 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.134432    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9fe5a2e-bdf4-4461-a966-d9f47fc786a8-must-gather-output\") pod \"must-gather-ngsx7\" (UID: \"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8\") " pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.146335 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.146316    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgpvh\" (UniqueName: \"kubernetes.io/projected/a9fe5a2e-bdf4-4461-a966-d9f47fc786a8-kube-api-access-fgpvh\") pod \"must-gather-ngsx7\" (UID: \"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8\") " pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.254094 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.254021    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7f9m7/must-gather-ngsx7"
Jul 19 14:56:43.406128 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:43.406103    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7f9m7/must-gather-ngsx7"]
Jul 19 14:56:43.407311 ip-10-0-143-90 kubenswrapper[2574]: W0719 14:56:43.407288    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9fe5a2e_bdf4_4461_a966_d9f47fc786a8.slice/crio-728f40820ed7cd3865fc5be27f564b98d551af4be3695a01fa45f33fab0733cf WatchSource:0}: Error finding container 728f40820ed7cd3865fc5be27f564b98d551af4be3695a01fa45f33fab0733cf: Status 404 returned error can't find the container with id 728f40820ed7cd3865fc5be27f564b98d551af4be3695a01fa45f33fab0733cf
Jul 19 14:56:44.399627 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:44.399488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f9m7/must-gather-ngsx7" event={"ID":"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8","Type":"ContainerStarted","Data":"3ff199f9f1eb910689bed201860f95fa9b9d7d9b3c072038cf64148037b54ff8"}
Jul 19 14:56:44.399627 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:44.399580    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f9m7/must-gather-ngsx7" event={"ID":"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8","Type":"ContainerStarted","Data":"728f40820ed7cd3865fc5be27f564b98d551af4be3695a01fa45f33fab0733cf"}
Jul 19 14:56:45.406008 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:45.405971    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f9m7/must-gather-ngsx7" event={"ID":"a9fe5a2e-bdf4-4461-a966-d9f47fc786a8","Type":"ContainerStarted","Data":"62beaef85fb461d42bb1f3e2ec5bf0db7f0f906ac7f387315a5cf4ff4f1eeed5"}
Jul 19 14:56:45.437678 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:45.437629    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7f9m7/must-gather-ngsx7" podStartSLOduration=2.62432728 podStartE2EDuration="3.437612864s" totalImagesPullingTime="813.285584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:45.436255164 +0000 UTC m=+2623.739086180" watchObservedRunningTime="2026-07-19 14:56:45.437612864 +0000 UTC m=+2623.740443904"
Jul 19 14:56:46.780663 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:46.780635    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x7d24_ced506fe-e66e-4a7f-9fed-f1c72377b3ec/global-pull-secret-syncer/0.log"
Jul 19 14:56:46.942691 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:46.942640    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log"
Jul 19 14:56:47.603793 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:47.603736    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-90.ec2.internal_eabed4ba2b3d6a2cc98e02bb7bec66ad/haproxy/0.log"
Jul 19 14:56:48.355282 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:48.355242    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:48.615387 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:48.615345    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mfrj_cda87231-5ef5-41b0-b0be-6ae5faff9144/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:55.015369 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.015332    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-78hlt_768c69be-5675-444e-887f-18be8288fdd6/kube-state-metrics/0.log"
Jul 19 14:56:55.073341 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.073313    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-78hlt_768c69be-5675-444e-887f-18be8288fdd6/kube-rbac-proxy-main/0.log"
Jul 19 14:56:55.121143 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.121116    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-78hlt_768c69be-5675-444e-887f-18be8288fdd6/kube-rbac-proxy-self/0.log"
Jul 19 14:56:55.172361 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.172334    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67b86fb4c4-9xsn2_19861593-6358-4110-8ae1-2e8d1454956a/metrics-server/0.log"
Jul 19 14:56:55.323179 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.323057    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"]
Jul 19 14:56:55.328926 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.328896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.341862 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.341828    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"]
Jul 19 14:56:55.457991 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.457952    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9p6\" (UniqueName: \"kubernetes.io/projected/8442e8b6-7a38-4179-b655-9b06d91fdd41-kube-api-access-pj9p6\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.458462 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.458442    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-proc\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.458685 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.458659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-lib-modules\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.458808 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.458691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-podres\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.458808 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.458785    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-sys\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.492499 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.492470    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bxbmz_5fde7a40-2455-4de1-afba-fb2cce517190/node-exporter/0.log"
Jul 19 14:56:55.529109 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.529075    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bxbmz_5fde7a40-2455-4de1-afba-fb2cce517190/kube-rbac-proxy/0.log"
Jul 19 14:56:55.559952 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.559917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-lib-modules\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.559952 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.559959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-podres\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-sys\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-lib-modules\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9p6\" (UniqueName: \"kubernetes.io/projected/8442e8b6-7a38-4179-b655-9b06d91fdd41-kube-api-access-pj9p6\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560163    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-podres\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560211 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-proc\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560442 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560217    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-sys\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.560442 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.560270    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8442e8b6-7a38-4179-b655-9b06d91fdd41-proc\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.567162 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.567141    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bxbmz_5fde7a40-2455-4de1-afba-fb2cce517190/init-textfile/0.log"
Jul 19 14:56:55.581648 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.581580    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9p6\" (UniqueName: \"kubernetes.io/projected/8442e8b6-7a38-4179-b655-9b06d91fdd41-kube-api-access-pj9p6\") pod \"perf-node-gather-daemonset-wpmb9\" (UID: \"8442e8b6-7a38-4179-b655-9b06d91fdd41\") " pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.642703 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.642667    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:55.808584 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.808549    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"]
Jul 19 14:56:55.832029 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.831943    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zmq6v_9028259e-9088-4c2c-89ce-79a693a6dc61/kube-rbac-proxy-main/0.log"
Jul 19 14:56:55.861328 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.861302    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zmq6v_9028259e-9088-4c2c-89ce-79a693a6dc61/kube-rbac-proxy-self/0.log"
Jul 19 14:56:55.890018 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.889991    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zmq6v_9028259e-9088-4c2c-89ce-79a693a6dc61/openshift-state-metrics/0.log"
Jul 19 14:56:55.924575 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.924549    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/prometheus/0.log"
Jul 19 14:56:55.953685 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.953662    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/config-reloader/0.log"
Jul 19 14:56:55.982253 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:55.982227    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/thanos-sidecar/0.log"
Jul 19 14:56:56.012584 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.012550    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/kube-rbac-proxy-web/0.log"
Jul 19 14:56:56.042053 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.042026    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/kube-rbac-proxy/0.log"
Jul 19 14:56:56.069612 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.069583    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/kube-rbac-proxy-thanos/0.log"
Jul 19 14:56:56.100128 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.100052    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_530ce3f8-0b31-46e0-a4e2-f27ac320555d/init-config-reloader/0.log"
Jul 19 14:56:56.133014 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.132983    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-z25rn_5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee/prometheus-operator/0.log"
Jul 19 14:56:56.162465 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.162428    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-z25rn_5408a7b9-89d1-4e9e-ac91-2ed4ce6bfbee/kube-rbac-proxy/0.log"
Jul 19 14:56:56.267812 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.267776    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b88665b5f-thcfn_21106d58-e2cd-4037-b7ce-3d1c0d23d3ec/telemeter-client/0.log"
Jul 19 14:56:56.306130 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.306099    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b88665b5f-thcfn_21106d58-e2cd-4037-b7ce-3d1c0d23d3ec/reload/0.log"
Jul 19 14:56:56.350581 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.350498    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b88665b5f-thcfn_21106d58-e2cd-4037-b7ce-3d1c0d23d3ec/kube-rbac-proxy/0.log"
Jul 19 14:56:56.485542 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.485487    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9" event={"ID":"8442e8b6-7a38-4179-b655-9b06d91fdd41","Type":"ContainerStarted","Data":"01993643a08df59c5adf299cc0d2c194648a5158d7fc083af0d87797dcdd4a7c"}
Jul 19 14:56:56.485730 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.485564    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9" event={"ID":"8442e8b6-7a38-4179-b655-9b06d91fdd41","Type":"ContainerStarted","Data":"ae742c3ba9958116e75f5e7f27b5bd38a0e6c00a09da8cca397d4974de961f8b"}
Jul 19 14:56:56.485730 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.485701    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:56:56.527021 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:56.526961    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9" podStartSLOduration=1.5269354910000001 podStartE2EDuration="1.526935491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:56.525652833 +0000 UTC m=+2634.828483853" watchObservedRunningTime="2026-07-19 14:56:56.526935491 +0000 UTC m=+2634.829766510"
Jul 19 14:56:58.795440 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:56:58.795407    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-zbb8r_7447d118-fa0c-4fa0-829d-cc6f2ad4d67c/download-server/0.log"
Jul 19 14:57:00.529784 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:00.529747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8795k_efc9cd0e-1194-4797-a2b8-2ef8092c1985/dns/0.log"
Jul 19 14:57:00.555836 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:00.555812    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8795k_efc9cd0e-1194-4797-a2b8-2ef8092c1985/kube-rbac-proxy/0.log"
Jul 19 14:57:00.683763 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:00.683724    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-597hr_76b4d6b2-022a-48a5-a1ca-195f08d99cf3/dns-node-resolver/0.log"
Jul 19 14:57:01.343869 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:01.343830    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6dd8cfb444-cvsqf_fb814f35-ecd0-427d-8b1c-e7851a01f734/registry/0.log"
Jul 19 14:57:01.393427 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:01.393390    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kxf6j_42db642d-3fb7-4e9c-a3f2-e726a97bf514/node-ca/0.log"
Jul 19 14:57:02.503362 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:02.503333    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7f9m7/perf-node-gather-daemonset-wpmb9"
Jul 19 14:57:02.560718 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:02.560692    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf8c8f977-grvz4_5379fabd-911c-4adc-ab08-27d4f6367d33/kube-auth-proxy/0.log"
Jul 19 14:57:02.683392 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:02.683318    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jrzjx_841db817-1a2c-4981-99e3-31c9b3d10f98/payload-pre-processing/0.log"
Jul 19 14:57:02.718226 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:02.718171    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ndxvw_79e0cc04-0363-4041-a1e6-aafb1513b74e/payload-processing/0.log"
Jul 19 14:57:03.530295 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:03.530263    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wrm4x_665147b0-a844-4f47-a7db-38498a65ea47/serve-healthcheck-canary/0.log"
Jul 19 14:57:04.213252 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:04.213176    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqqjj_187bc853-e9de-411a-a649-f239f8f3dcfc/kube-rbac-proxy/0.log"
Jul 19 14:57:04.237948 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:04.237915    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqqjj_187bc853-e9de-411a-a649-f239f8f3dcfc/exporter/0.log"
Jul 19 14:57:04.262245 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:04.262218    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqqjj_187bc853-e9de-411a-a649-f239f8f3dcfc/extractor/0.log"
Jul 19 14:57:06.234174 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:06.234146    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf8c8f977-grvz4_5379fabd-911c-4adc-ab08-27d4f6367d33/kube-auth-proxy/0.log"
Jul 19 14:57:06.348799 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:06.348769    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jrzjx_841db817-1a2c-4981-99e3-31c9b3d10f98/payload-pre-processing/0.log"
Jul 19 14:57:06.382394 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:06.382363    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ndxvw_79e0cc04-0363-4041-a1e6-aafb1513b74e/payload-processing/0.log"
Jul 19 14:57:07.199420 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:07.198087    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zhkx9_78f989a3-2735-4546-b6f2-1c7f82c08499/manager/0.log"
Jul 19 14:57:07.424346 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:07.424306    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57866856f9-ztrk5_3a37e812-ea38-4904-b2b1-995cfac53432/manager/0.log"
Jul 19 14:57:07.775816 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:07.775786    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bd7c6769d-62tfq_8f85a9ec-654f-4c7b-ba7f-95ae715f41bd/manager/0.log"
Jul 19 14:57:07.871997 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:07.871969    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bd7c6769d-62tfq_8f85a9ec-654f-4c7b-ba7f-95ae715f41bd/copy-manifests/0.log"
Jul 19 14:57:07.991083 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:07.991054    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf8c8f977-grvz4_5379fabd-911c-4adc-ab08-27d4f6367d33/kube-auth-proxy/0.log"
Jul 19 14:57:08.170128 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:08.170050    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-jrzjx_841db817-1a2c-4981-99e3-31c9b3d10f98/payload-pre-processing/0.log"
Jul 19 14:57:08.201276 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:08.201244    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ndxvw_79e0cc04-0363-4041-a1e6-aafb1513b74e/payload-processing/0.log"
Jul 19 14:57:09.619576 ip-10-0-143-90 kubenswrapper[2574]: I0719 14:57:09.619540    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7d56c9879c-tph6q_a93d13a3-027d-4269-b12d-37859e1b87cc/manager/0.log"